diff --git a/text/0000-windows-subsystem.md b/text/0000-windows-subsystem.md new file mode 100644 index 00000000000..3de857c72de --- /dev/null +++ b/text/0000-windows-subsystem.md @@ -0,0 +1,161 @@ +- Feature Name: Windows Subsystem +- Start Date: 2016-07-03 +- RFC PR: ____ +- Rust Issue: ____ + +# Summary +[summary]: #summary + +Rust programs compiled for windows will always allocate a console window on +startup. This behavior is controlled via the `SUBSYSTEM` parameter passed to the +linker, and so *can* be overridden with specific compiler flags. However, doing +so will bypass the rust-specific initialization code in `libstd`, as when using +the MSVC toolchain, the entry point must be named `WinMain`. + +This RFC proposes supporting this case explicitly, allowing `libstd` to +continue to be initialized correctly. + +# Motivation +[motivation]: #motivation + +The `WINDOWS` subsystem is commonly used on windows: desktop applications +typically do not want to flash up a console window on startup. + +Currently, using the `WINDOWS` subsystem from rust is undocumented, and the +process is non-trivial when targeting the MSVC toolchain. There are a couple of +approaches, each with their own downsides: + +## Define a WinMain symbol + +A new symbol `pub extern "system" WinMain(...)` with specific argument +and return types must be declared, which will become the new entry point for +the program. + +This is unsafe, and will skip the initialization code in `libstd`. + +The GNU toolchain will accept either entry point. + +## Override the entry point via linker options + +This uses the same method as will be described in this RFC. However, it will +result in build scripts also being compiled for the windows subsystem, which +can cause additional console windows to pop up during compilation, making the +system unusable while a build is in progress. + +# Detailed design +[design]: #detailed-design + +When an executable is linked while compiling for a windows target, it will be +linked for a specific *subsystem*. The subsystem determines how the operating +system will run the executable, and will affect the execution environment of +the program. + +In practice, only two subsystems are very commonly used: `CONSOLE` and +`WINDOWS`, and from a user's perspective, they determine whether a console will +be automatically created when the program is started. + +## New crate attribute + +This RFC proposes two changes to solve this problem. The first is adding a +top-level crate attribute to allow specifying which subsystem to use: + +`#![windows_subsystem = "windows"]` + +Initially, the set of possible values will be `{windows, console}`, but may be +extended in future if desired. + +The use of this attribute in a non-executable crate will result in a compiler +warning. If compiling for a non-windows target, the attribute will be silently +ignored. + +## Additional linker argument + +For the GNU toolchain, this will be sufficient. However, for the MSVC toolchain, +the linker will be expecting a `WinMain` symbol, which will not exist. + +There is some complexity to the way in which a different entry point is expected +when using the windows subsystem. Firstly, the C-runtime library exports two +symbols designed to be used as an entry point: +``` +mainCRTStartup +WinMainCRTStartup +``` + +`LINK.exe` will use the subsystem to determine which of these symbols to use +as the default entry point if not overridden. + +Each one performs some unspecified initialization of the CRT, before calling out +to a symbol defined within the program (`main` or `WinMain` respectively). + +The second part of the solution is to pass an additional linker option when +targeting the MSVC toolchain: +`/ENTRY:mainCRTStartup` + +This will override the entry point to always be `mainCRTStartup`. For +console-subsystem programs this will have no effect, since it was already the +default, but for windows-subsystem programs, it will eliminate the need for +a `WinMain` symbol to be defined. + +This command line option will always be passed to the linker, regardless of the +presence or absence of the `windows_subsystem` crate attribute, except when +the user specifies their own entry point in the linker arguments. This will +require `rustc` to perform some basic parsing of the linker options. + +# Drawbacks +[drawbacks]: #drawbacks + +- A new platform-specific crate attribute. +- The difficulty of manually calling the rust initialization code is potentially + a more general problem, and this only solves a specific (if common) case. +- The subsystem must be specified earlier than is strictly required: when + compiling C/C++ code only the linker, not the compiler, needs to actually be + aware of the subsystem. +- It is assumed that the initialization performed by the two CRT entry points + is identical. This seems to currently be the case, and is unlikely to change + as this technique appears to be used fairly widely. + +# Alternatives +[alternatives]: #alternatives + +- Only emit one of either `WinMain` or `main` from `rustc` based on a new + command line option. + + This command line option would only be applicable when compiling an + executable, and only for windows platforms. No other supported platforms + require a different entry point or additional linker arguments for programs + designed to run with a graphical user interface. + + `rustc` will react to this command line option by changing the exported + name of the entry point to `WinMain`, and passing additional arguments to + the linker to configure the correct subsystem. A mismatch here would result + in linker errors. + + A similar option would need to be added to `Cargo.toml` to make usage as + simple as possible. + + There's some bike-shedding which can be done one the exact command line + interface, but one possible option is shown below. + + Rustc usage: + `rustc foo.rs --crate-subsystem windows` + + Cargo.toml + ```toml + [package] + # ... + + [[bin]] + name = "foo" + path = "src/foo.rs" + subsystem = "windows" + ``` + + The `crate-subsystem` command line option would exist on all platforms, + but would be ignored when compiling for a non-windows target, so as to + support cross-compiling. If not compiling a binary crate, specifying the + option is an error regardless of the target. + +# Unresolved questions +[unresolved]: #unresolved-questions + +None