From 6301bcc57aa664470178f38b7ef70c5124d0ecdd Mon Sep 17 00:00:00 2001 From: Alex Crichton Date: Mon, 23 Feb 2015 15:48:37 -0800 Subject: [PATCH 1/6] Amend RFC 517: Add material for stdio Expand the section on stdin, stdout, and stderr while also adding a new section explaining the fate of the current print-related functions. --- text/0517-io-os-reform.md | 49 ++++++++++++++++++++++++++++++++++++--- 1 file changed, 46 insertions(+), 3 deletions(-) diff --git a/text/0517-io-os-reform.md b/text/0517-io-os-reform.md index a867e549266..8bd44018d3b 100644 --- a/text/0517-io-os-reform.md +++ b/text/0517-io-os-reform.md @@ -62,6 +62,7 @@ follow-up PRs against this RFC. * [Errors] * [Channel adapters] * [stdin, stdout, stderr] + * [Printing functions] * [std::env] * [std::fs] * [Free functions] @@ -72,7 +73,6 @@ follow-up PRs against this RFC. * [TCP] * [UDP] * [Addresses] - * [std::net] (stub) * [std::process] * [Command] * [Child] @@ -1155,7 +1155,49 @@ RFC recommends they remain unstable. #### `stdin`, `stdout`, `stderr` [stdin, stdout, stderr]: #stdin-stdout-stderr -> To be added in a follow-up PR. +The current `stdio` module will be removed in favor of three constructors: + +* `stdin` - returns a handle to a **globally shared** to the standard input of + the process which is buffered as well. All operations on this handle will + first require acquiring a lock to ensure access to the shared buffer is + synchronized. The handle can be explicitly locked for a critical section so + relocking is not necessary. + + The `Read` trait will be implemented directly on the returned `Stdin` handle + but the `BufRead` trait will not be (due to synchronization concerns). The + locked version of `Stdin` will provide an implementation of `BufRead`. + + The design will largely be the same as is today with the `old_io` module. + +* `stderr` - returns a **non buffered** handle to the standard error output + stream for the process. Each call to `write` will roughly translate to a + system call to output data when written to `stderr`. + +* `stdout` - returns a **locally buffered** handle to the standard output of the + current process. The amount of buffering can be decided at runtime to allow + for different situations such as being attached to a TTY or being redirected + to an output file. The `Write` trait will be implemented for this handle. + +The `stderr_raw` constructor is removed because the handle is no longer buffered +and the `stdin_raw` and `stdout_raw` handles are removed to be added at a later +date in the `std::os` modules if necessary. + +#### Printing functions +[Printing functions]: #printing-functions + +The current `print`, `println`, `print_args`, and `println_args` functions will +all be "removed from the public interface" by [prefixing them with `__` and +marking `#[doc(hidden)]`][gh22607]. These are all implementation details of the +`print!` and `println!` macros and don't need to be exposed in the public +interface. + +[gh22607]: https://github.com/rust-lang/rust/issues/22607 + +The `set_stdout` and `set_stderr` functions will be moved to a new +`std::fmt::output` module and renamed to `set_print` and `set_panic`, +respectively. These new names reflect what they actually do, removing a +longstanding confusion. The current `stdio::flush` function will also move to +this module and be renamed to `flush_print`. ### `std::env` [std::env]: #stdenv @@ -1173,7 +1215,8 @@ and the signatures will be updated to follow this RFC's * `vars` (renamed from `env`): yields a vector of `(OsString, OsString)` pairs. * `var` (renamed from `getenv`): take a value bounded by `AsOsStr`, - allowing Rust strings and slices to be ergonomically passed in. Yields an `Option`. + allowing Rust strings and slices to be ergonomically passed in. Yields an + `Option`. * `var_string`: take a value bounded by `AsOsStr`, returning `Result` where `VarError` represents a non-unicode `OsString` or a "not present" value. From 0466d1836362d275fd015b44cb79aedab3b8174f Mon Sep 17 00:00:00 2001 From: Alex Crichton Date: Tue, 24 Feb 2015 21:44:08 -0800 Subject: [PATCH 2/6] All handles are globally locked, stdout is globally buffered Also add back text for `foo_raw` and details. --- text/0517-io-os-reform.md | 74 +++++++++++++++++++++++++++++++++------ 1 file changed, 64 insertions(+), 10 deletions(-) diff --git a/text/0517-io-os-reform.md b/text/0517-io-os-reform.md index 8bd44018d3b..bb178255b95 100644 --- a/text/0517-io-os-reform.md +++ b/text/0517-io-os-reform.md @@ -1155,9 +1155,19 @@ RFC recommends they remain unstable. #### `stdin`, `stdout`, `stderr` [stdin, stdout, stderr]: #stdin-stdout-stderr -The current `stdio` module will be removed in favor of three constructors: +The current `stdio` module will be removed in favor of these constructors in the +`io` module: -* `stdin` - returns a handle to a **globally shared** to the standard input of +```rust +fn stdin() -> Stdin; +fn stdout() -> Stdout; +fn stderr() -> Stderr; +fn stdin_raw() -> StdinRaw; +fn stdout_raw() -> StdoutRaw; +fn stderr_raw() -> StderrRaw; +``` + +* `stdin` - returns a handle to a **globally shared** standard input of the process which is buffered as well. All operations on this handle will first require acquiring a lock to ensure access to the shared buffer is synchronized. The handle can be explicitly locked for a critical section so @@ -1169,18 +1179,60 @@ The current `stdio` module will be removed in favor of three constructors: The design will largely be the same as is today with the `old_io` module. + ```rust + impl Stdin { + fn lock(&self) -> StdinLock; + fn read_line(&mut self, into: &mut String) -> io::Result<()>; + fn read_until(&mut self, byte: u8, into: &mut Vec) -> io::Result<()>; + } + impl Read for Stdin { ... } + impl Read for StdinLock { ... } + impl BufRead for StdinLock { ... } + ``` + * `stderr` - returns a **non buffered** handle to the standard error output stream for the process. Each call to `write` will roughly translate to a - system call to output data when written to `stderr`. + system call to output data when written to `stderr`. This handle is locked + like `stdin` to ensure, for example, that calls to `write_all` are atomic with + respect to one another. There will also be an RAII guard to lock the handle + and use the result as an instance of `Write`. -* `stdout` - returns a **locally buffered** handle to the standard output of the - current process. The amount of buffering can be decided at runtime to allow - for different situations such as being attached to a TTY or being redirected - to an output file. The `Write` trait will be implemented for this handle. + ```rust + impl Stderr { + fn lock(&self) -> StderrLock; + } + impl Write for Stderr { ... } + impl Write for StderrLock { ... } + ``` -The `stderr_raw` constructor is removed because the handle is no longer buffered -and the `stdin_raw` and `stdout_raw` handles are removed to be added at a later -date in the `std::os` modules if necessary. +* `stdout` - returns a **globally buffered** handle to the standard output of + the current process. The amount of buffering can be decided at runtime to + allow for different situations such as being attached to a TTY or being + redirected to an output file. The `Write` trait will be implemented for this + handle, and like `stderr` it will be possible to lock it and then use the + result as an instance of `Write` as well. + + ```rust + impl Stdout { + fn lock(&self) -> StdoutLock; + } + impl Write for Stdout { ... } + impl Write for StdoutLock { ... } + ``` + +* `*_raw` - these constructors will return references to the raw handles which + are guaranteed to not be protected with any form of lock or have any backing + buffer. Their APIs will look like: + + ```rust + impl Read for StdinRaw { ... } + impl Write for StdoutRaw { ... } + impl Write for StderrRaw { ... } + ``` + + The documentation for `stdin_raw` will indicate that extra data may be + buffered in the `stdin` handle which will not be accessible to the `stdin_raw` + handle. #### Printing functions [Printing functions]: #printing-functions @@ -1199,6 +1251,8 @@ respectively. These new names reflect what they actually do, removing a longstanding confusion. The current `stdio::flush` function will also move to this module and be renamed to `flush_print`. +The entire `std::fmt::output` module will remain `#[unstable]` for now, however. + ### `std::env` [std::env]: #stdenv From ea2b0e79e08ae0ab84277ce155b9885641b38445 Mon Sep 17 00:00:00 2001 From: Alex Crichton Date: Wed, 25 Feb 2015 13:59:49 -0800 Subject: [PATCH 3/6] Clarify stdio_raw and behavior on Windows vs Unix --- text/0517-io-os-reform.md | 103 +++++++++++++++++++++++++++++++------- 1 file changed, 86 insertions(+), 17 deletions(-) diff --git a/text/0517-io-os-reform.md b/text/0517-io-os-reform.md index bb178255b95..0f648c2f55c 100644 --- a/text/0517-io-os-reform.md +++ b/text/0517-io-os-reform.md @@ -1159,12 +1159,9 @@ The current `stdio` module will be removed in favor of these constructors in the `io` module: ```rust -fn stdin() -> Stdin; -fn stdout() -> Stdout; -fn stderr() -> Stderr; -fn stdin_raw() -> StdinRaw; -fn stdout_raw() -> StdoutRaw; -fn stderr_raw() -> StderrRaw; +pub fn stdin() -> Stdin; +pub fn stdout() -> Stdout; +pub fn stderr() -> Stderr; ``` * `stdin` - returns a handle to a **globally shared** standard input of @@ -1220,19 +1217,91 @@ fn stderr_raw() -> StderrRaw; impl Write for StdoutLock { ... } ``` -* `*_raw` - these constructors will return references to the raw handles which - are guaranteed to not be protected with any form of lock or have any backing - buffer. Their APIs will look like: +#### Windows and stdio +[Windows stdio]: #windows-and-stdio - ```rust - impl Read for StdinRaw { ... } - impl Write for StdoutRaw { ... } - impl Write for StderrRaw { ... } - ``` +On Windows, standard input and output handles can work with either arbitrary +`[u8]` or `[u16]` depending on the state at runtime. For example a program +attached to the console will work with arbitrary `[u16]`, but a program attached +to a pipe would work with arbitrary `[u8]`. + +To handle this difference, the following behavior will be enforced for the +standard primitives listed above: + +* If attached to a pipe then no attempts at encoding or decoding will be done, + the data will be ferried through as `[u8]`. + +* If attached to a console, then `stdin` will attempt to interpret all input as + UTF-16, re-encoding into UTF-8 and returning the UTF-8 data instead. This + implies that data will be buffered internally to handle partial reads/writes. + Invalid UTF-16 will simply be discarded returning an `io::Error` explaining + why. + +* If attached to a console, then `stdout` and `stderr` will attempt to interpret + input as UTF-8, re-encoding to UTF-16. If the input is not valid UTF-8 then an + error will be returned and no data will be written. + +#### Raw stdio +[Raw stdio]: #raw-stdio + +The above standard input/output handles all involve some form of locking or +buffering (or both). This cost is not always wanted, and hence raw variants will +be provided. Due to platform differences across unix/windows, the following +structure will be supported: + +```rust +mod os { + mod unix { + mod stdio { + struct Stdio { .. } + + impl Stdio { + fn stdout() -> Stdio; + fn stderr() -> Stdio; + fn stdin() -> Stdio; + } + + impl Read for Stdio { ... } + impl Write for Stdio { ... } + } + } + + mod windows { + mod stdio { + struct Stdio { ... } + struct StdioConsole { ... } + + impl Stdio { + fn stdout() -> io::Result; + fn stderr() -> io::Result; + fn stdin() -> io::Result; + } + // same constructors StdioConsole + + impl Read for Stdio { ... } + impl Write for Stdio { ... } + + impl StdioConsole { + // returns slice of what was read + fn read<'a>(&self, buf: &'a mut OsString) -> io::Result<&'a OsStr>; + // returns remaining part of `buf` to be written + fn write<'a>(&self, buf: &'a OsStr) -> io::Result<&'a OsStr>; + } + } + } +} +``` + +There are some key differences from today's API: - The documentation for `stdin_raw` will indicate that extra data may be - buffered in the `stdin` handle which will not be accessible to the `stdin_raw` - handle. +* On unix, the API has not changed much except that the handles have been + consolidated into one type which implements both `Read` and `Write` (although + writing to stdin is likely to generate an error). +* On windows, there are two sets of handles representing the difference between + "console mode" and not (e.g. a pipe). When not a console the normal I/O traits + are implemented (delegating to `ReadFile` and `WriteFile`. The console mode + operations work with `OsStr`, however, to show how they work with UCS-2 under + the hood. #### Printing functions [Printing functions]: #printing-functions From f3be78b317cbd71223ec1a99527f41e096feadf5 Mon Sep 17 00:00:00 2001 From: Alex Crichton Date: Wed, 4 Mar 2015 16:30:44 -0800 Subject: [PATCH 4/6] Clarify raw stdio will not get implemented just yet --- text/0517-io-os-reform.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/text/0517-io-os-reform.md b/text/0517-io-os-reform.md index 0f648c2f55c..c8e73839e68 100644 --- a/text/0517-io-os-reform.md +++ b/text/0517-io-os-reform.md @@ -1244,6 +1244,10 @@ standard primitives listed above: #### Raw stdio [Raw stdio]: #raw-stdio +> **Note**: This section is intended to be a sketch of possible raw stdio +> support, but it is not planned to implement or stabilize this +> implementation at this time. + The above standard input/output handles all involve some form of locking or buffering (or both). This cost is not always wanted, and hence raw variants will be provided. Due to platform differences across unix/windows, the following From 5b569fc7189bd128fb8e108cb0ce1aa9d4a511a6 Mon Sep 17 00:00:00 2001 From: Alex Crichton Date: Wed, 4 Mar 2015 16:43:42 -0800 Subject: [PATCH 5/6] Remove `std::fmt::output` --- text/0517-io-os-reform.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/text/0517-io-os-reform.md b/text/0517-io-os-reform.md index c8e73839e68..0b80808821f 100644 --- a/text/0517-io-os-reform.md +++ b/text/0517-io-os-reform.md @@ -1318,13 +1318,11 @@ interface. [gh22607]: https://github.com/rust-lang/rust/issues/22607 -The `set_stdout` and `set_stderr` functions will be moved to a new -`std::fmt::output` module and renamed to `set_print` and `set_panic`, -respectively. These new names reflect what they actually do, removing a -longstanding confusion. The current `stdio::flush` function will also move to -this module and be renamed to `flush_print`. - -The entire `std::fmt::output` module will remain `#[unstable]` for now, however. +The `set_stdout` and `set_stderr` functions will be removed with no replacement +for now. It's unclear whether these functions should indeed control a thread +local handle instead of a global handle as whether they're justified in the +first place. It is a backwards-compatible extension to allow this sort of output +to be redirected and can be considered if the need arises. ### `std::env` [std::env]: #stdenv From 48fbc1456f0e3fae29a10c217559908819e44a2a Mon Sep 17 00:00:00 2001 From: Alex Crichton Date: Tue, 10 Mar 2015 10:30:47 -0700 Subject: [PATCH 6/6] Clarify wording around locking --- text/0517-io-os-reform.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/text/0517-io-os-reform.md b/text/0517-io-os-reform.md index 0b80808821f..6babd82b337 100644 --- a/text/0517-io-os-reform.md +++ b/text/0517-io-os-reform.md @@ -1165,14 +1165,16 @@ pub fn stderr() -> Stderr; ``` * `stdin` - returns a handle to a **globally shared** standard input of - the process which is buffered as well. All operations on this handle will - first require acquiring a lock to ensure access to the shared buffer is - synchronized. The handle can be explicitly locked for a critical section so - relocking is not necessary. + the process which is buffered as well. Due to the globally shared nature of + this handle, all operations on `Stdin` directly will acquire a lock internally + to ensure access to the shared buffer is synchronized. This implementation + detail is also exposed through a `lock` method where the handle can be + explicitly locked for a period of time so relocking is not necessary. The `Read` trait will be implemented directly on the returned `Stdin` handle but the `BufRead` trait will not be (due to synchronization concerns). The - locked version of `Stdin` will provide an implementation of `BufRead`. + locked version of `Stdin` (`StdinLock`) will provide an implementation of + `BufRead`. The design will largely be the same as is today with the `old_io` module.