From f4ca080b14e1161ff24a5df8c2a897a96fc63e1f Mon Sep 17 00:00:00 2001 From: Michael Gattozzi Date: Wed, 26 Apr 2017 11:02:16 -0400 Subject: [PATCH 1/6] Add external doc attribute to rustc This RFC proposes an addition to be able to pull in documentation for items from an external source at compile time, rather than requiring them to be in the source code itself. --- text/0000-external-doc-attribute.md | 210 ++++++++++++++++++++++++++++ 1 file changed, 210 insertions(+) create mode 100644 text/0000-external-doc-attribute.md diff --git a/text/0000-external-doc-attribute.md b/text/0000-external-doc-attribute.md new file mode 100644 index 00000000000..ecde3b13f28 --- /dev/null +++ b/text/0000-external-doc-attribute.md @@ -0,0 +1,210 @@ +- Feature Name: external_doc +- Start Date: 2017-04-26 +- RFC PR: (leave this empty) +- Rust Issue: (leave this empty) + +# Summary +[summary]: #summary + +Documentation is an important part of any project, it allows developers to +explain how to use items within a library as well as communicate the intent of +how to use it through examples. Rust has long championed this feature through +the use of documentation comments and `rustdoc` to generate beautiful, easy to +navigate documentation. However, there is no way right now to have documentation +be imported into the code from an external file. This RFC proposes a way to +extend the functionality of Rust to include this ability. + +# Motivation +[motivation]: #motivation + +1. Many smaller crates are able to do all of the documentation that's needed in + a README file within their repo. Being able to include this as a crate or + module level doc comment would mean not having to duplicate documentation and + is easier to maintain. This means that one could run `cargo doc` with the + small crate as a dependency and be able to access the contents of the README + without needing to go online to the repo to read it. This also would help + with [this issue on + crates.io](https://github.com/rust-lang/crates.io/issues.81) by making it + easy to have the README in the crate and the crate root at the same. +2. The feature would provide a way to have easier to read code for library + maintainers. Sometimes doc comments are quite long in terms of line count + (items in + [libstd](https://github.com/rust-lang/rust/blob/master/src/libstd) are a good + example of this). Doc comments document behavior of functions, structs, and + types to the end user, they do not explain for a coder working on the library + as to how they work internally. When actually writing code for a + library the doc comments end up cluttering the source code making it harder + to find relevant lines to change or skim through and read what is going on. +3. Localization is something else that would further open up access to the + community. By providing docs in different languages we could significantly + expand our reach as a community and be more inclusive of those where English + is not their first language. This would be made possible with a config flag + choosing what file to import as a doc comment. + +These are just a few reasons as to why we should do this, but the expected +outcome of this feature is expected to be positive with little to no downside +for a user. + +# Detailed design +[design]: #detailed-design + +All files included through the attribute will be relative paths from the crate +root directory. Given a file like this stored in `docs/example.md`: + +```md +# I'm an example +This is a markdown file that gets imported to Rust as a Doc comment. +``` + +And code like this: + +```rust +#[external_doc("docs/example.md", "line")] +fn my_func() { + // Hidden implementation +} +``` + +It should expand to this at compile time: + +```rust +/// # I'm an example +/// This is a markdown file that gets imported to Rust as a doc comment. +fn my_func() { + // Hidden implementation +} +``` + +Which `rustdoc` should be able to figure out and use for documentation. + +If the flag is changed from `line` to `mod` the file is then imported as +a module comment for the file (or module) the attribute exists in. +Using the previous markdown file we would have this code: + +```rust +#[external_doc("docs/example.md", "mod")] +fn my_func() { + // Hidden implementation +} +``` + +Expand to this at compile time: + +```rust +//! # I'm an example +//! This is a markdown file that gets imported to Rust as a doc comment. +fn my_func() { + // Hidden implementation +} +``` + +In the case of this code: + +```rust +mod example { + #[external_doc("docs/example.md", "mod")] + fn my_func() { + // Hidden implementation + } +} +``` + +It should expand out to: + +```rust +mod example { + //! # I'm an example + //! This is a markdown file that gets imported to Rust as a doc comment. + fn my_func() { + // Hidden implementation + } +} +``` + +## Line numbers when errors occur +As with all macros being expanded this brings up the question of line numbers +and for documentation tests especially so, to keep things simple for the user +the documentation should be treated separately from the code. Since the +attribute only needs to be expanded with `rustdoc` or `cargo test`, it should be +ignored by the compiler except for having the proper lines for error messages. + +For example if we have this: + +```rust +#[external_doc("docs/example.md", "line")] // Line 1 +f my_func() { // Line 2 + // Hidden implementation // Line 3 +} // Line 4 +``` + +Then we would have a syntax error on line 2, however the doc comment comes +before that. In this case the compiler would ignore the attribute for expansion, +but would say that the error occurs on line 2 rather than saying it is line 1 if +the attribute is ignored. This makes it easy for the user to spot their error. +This same behavior should be observed in the case of inline tests and those in +the tests directory. + +If we have a documentation test failure the line number should be for the +external doc file and the line number where it fails, rather than a line number +from the code base itself. Having the numbers for the lines being used because +they were inserted into the code for these scenarios would cause confusion and +would obfuscate where errors occur, making it harder not easier for end users, +making this feature useless if it creates ergonomic overhead like this. + +# How We Teach This +[how-we-teach-this]: #how-we-teach-this + +`#[external_doc("path", "type")]` is an extension of the current +`#[doc = "doc"]` attribute by allowing documentation to exist outside of the +source code. This isn't entirely hard to grasp if one is familiar with +attributes but if not then this syntax vs a `///` or `//!` type of comment +could cause confusion. By labeling the attribute as `external_doc`, having a +clear path and type (either `line` or `mod`) then should, at the very least, +provide context as to what's going on and where to find this file for inclusion. + +The acceptance of this proposal would minimally impact all levels of Rust users +as it is something that provides convenience but is not a necessary thing to +learn to use Rust. It should be taught to existing users by updating +documentation to show it in use and to include in in the Rust Programming +Language book to teach new users. Currently the newest version of The Rust +Programming Language does not include doc comments as part of it. This would +need to be expanded to explain how doc comments work in general and this new +syntax as well by showing that users can include docs from external sources. +The Rust Reference comments section would need to updated to include this new +syntax as well. + +# Drawbacks +[drawbacks]: #drawbacks + +- This might confuse or frustrate people reading the code directly who prefer + those doc comments to be inline with the code rather than in a separate file. + This creates a burden of ergonomics by having to know the context of the code + that the doc comment is for while reading it separately from the code it + documents. + +# Alternatives +[alternatives]: #alternatives + +Currently there already [exists a plugin](https://github.com/mgattozzi/rdoc) +that could be used as a reference and has shown that +[there is interest](https://www.reddit.com/r/rust/comments/67kqs6/announcing_rdoc_a_tiny_rustc_plugin_to_host_your/). +Some limitations though being that it did not have module doc support and it +would make doc test failures unclear as to where they happened, which could be +solved with better support and intrinsics from the compiler. + +This same idea could be implemented as a crate with procedural macros (which are +on nightly now) so that others can opt in to this rather than have it be part of +the language itself. Docs will remain the same as they always have and will +continue to work as is if this alternative is chosen, though this means we limit +what we do and do not want rustc/rustdoc to be able to achieve here when it +comes to docs. + +# Unresolved questions +[unresolved]: #unresolved-questions + +- Should we include multiple items in one file or should each type/field be + allowed one file? If so how do we delimit it for the parser? +- Are there corner cases not covered in the current design? If there are what + would be a good design to solve them? +- Is this the syntax we want to use for the attribute? Do we want to use + something else? From 2acd28a6ee97e0ac8da15c3d6619e3a29392cde6 Mon Sep 17 00:00:00 2001 From: Michael Gattozzi Date: Mon, 8 May 2017 15:21:00 -0400 Subject: [PATCH 2/6] Update syntax for external_docs --- text/0000-external-doc-attribute.md | 28 ++++++++++++---------------- 1 file changed, 12 insertions(+), 16 deletions(-) diff --git a/text/0000-external-doc-attribute.md b/text/0000-external-doc-attribute.md index ecde3b13f28..20a0a81fae8 100644 --- a/text/0000-external-doc-attribute.md +++ b/text/0000-external-doc-attribute.md @@ -59,7 +59,7 @@ This is a markdown file that gets imported to Rust as a Doc comment. And code like this: ```rust -#[external_doc("docs/example.md", "line")] +#[doc(include = "docs/example.md")] fn my_func() { // Hidden implementation } @@ -77,18 +77,16 @@ fn my_func() { Which `rustdoc` should be able to figure out and use for documentation. -If the flag is changed from `line` to `mod` the file is then imported as -a module comment for the file (or module) the attribute exists in. -Using the previous markdown file we would have this code: +If the code is written like this: ```rust -#[external_doc("docs/example.md", "mod")] +#![doc(include = "docs/example.md")] fn my_func() { // Hidden implementation } ``` -Expand to this at compile time: +It should expand out to this at compile time: ```rust //! # I'm an example @@ -102,7 +100,7 @@ In the case of this code: ```rust mod example { - #[external_doc("docs/example.md", "mod")] + #![doc(include = "docs/example.md")] fn my_func() { // Hidden implementation } @@ -131,10 +129,10 @@ ignored by the compiler except for having the proper lines for error messages. For example if we have this: ```rust -#[external_doc("docs/example.md", "line")] // Line 1 -f my_func() { // Line 2 - // Hidden implementation // Line 3 -} // Line 4 +#[doc(include = "docs/example.md")] // Line 1 +f my_func() { // Line 2 + // Hidden implementation // Line 3 +} // Line 4 ``` Then we would have a syntax error on line 2, however the doc comment comes @@ -167,11 +165,9 @@ as it is something that provides convenience but is not a necessary thing to learn to use Rust. It should be taught to existing users by updating documentation to show it in use and to include in in the Rust Programming Language book to teach new users. Currently the newest version of The Rust -Programming Language does not include doc comments as part of it. This would -need to be expanded to explain how doc comments work in general and this new -syntax as well by showing that users can include docs from external sources. -The Rust Reference comments section would need to updated to include this new -syntax as well. +Programming Language book has a section for [doc comments](https://doc.rust-lang.org/nightly/book/second-edition/ch14-02-publishing-to-crates-io.html#documentation-comments) that will need to be expanded +to show how users can include docs from external sources. The Rust Reference +comments section would need to updated to include this new syntax as well. # Drawbacks [drawbacks]: #drawbacks From 3a62b965b3ecdbb88a6054cab581347ff95534d6 Mon Sep 17 00:00:00 2001 From: Michael Gattozzi Date: Mon, 15 May 2017 17:17:09 -0400 Subject: [PATCH 3/6] Fix the old style still in RFC --- text/0000-external-doc-attribute.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-external-doc-attribute.md b/text/0000-external-doc-attribute.md index 20a0a81fae8..d1e9e8289d9 100644 --- a/text/0000-external-doc-attribute.md +++ b/text/0000-external-doc-attribute.md @@ -152,7 +152,7 @@ making this feature useless if it creates ergonomic overhead like this. # How We Teach This [how-we-teach-this]: #how-we-teach-this -`#[external_doc("path", "type")]` is an extension of the current +`#[doc(include = "file_path")]` is an extension of the current `#[doc = "doc"]` attribute by allowing documentation to exist outside of the source code. This isn't entirely hard to grasp if one is familiar with attributes but if not then this syntax vs a `///` or `//!` type of comment From 71a9e72e3b8061e2af4f95643ce31144179bef4c Mon Sep 17 00:00:00 2001 From: Michael Gattozzi Date: Mon, 24 Jul 2017 21:40:38 -0400 Subject: [PATCH 4/6] Add licensing to PR --- text/0000-external-doc-attribute.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/text/0000-external-doc-attribute.md b/text/0000-external-doc-attribute.md index d1e9e8289d9..e76b3e4c10a 100644 --- a/text/0000-external-doc-attribute.md +++ b/text/0000-external-doc-attribute.md @@ -1,3 +1,14 @@ + + - Feature Name: external_doc - Start Date: 2017-04-26 - RFC PR: (leave this empty) From 20eff8e920e6c314228f28c48c15b526685b268f Mon Sep 17 00:00:00 2001 From: Michael Gattozzi Date: Tue, 19 Sep 2017 22:44:07 -0400 Subject: [PATCH 5/6] FCP Updates --- text/0000-external-doc-attribute.md | 49 +++++++++++++++-------------- 1 file changed, 26 insertions(+), 23 deletions(-) diff --git a/text/0000-external-doc-attribute.md b/text/0000-external-doc-attribute.md index e76b3e4c10a..e220c58de29 100644 --- a/text/0000-external-doc-attribute.md +++ b/text/0000-external-doc-attribute.md @@ -56,7 +56,7 @@ These are just a few reasons as to why we should do this, but the expected outcome of this feature is expected to be positive with little to no downside for a user. -# Detailed design +# Detailed Design [design]: #detailed-design All files included through the attribute will be relative paths from the crate @@ -66,11 +66,10 @@ root directory. Given a file like this stored in `docs/example.md`: # I'm an example This is a markdown file that gets imported to Rust as a Doc comment. ``` - -And code like this: +where `src` is in the same directory as `docs`. Given code like this: ```rust -#[doc(include = "docs/example.md")] +#[doc(include = "../docs/example.md")] fn my_func() { // Hidden implementation } @@ -79,8 +78,7 @@ fn my_func() { It should expand to this at compile time: ```rust -/// # I'm an example -/// This is a markdown file that gets imported to Rust as a doc comment. +#[doc("# I'm an example\nThis is a markdown file that gets imported to Rust as a doc comment.")] fn my_func() { // Hidden implementation } @@ -91,7 +89,7 @@ Which `rustdoc` should be able to figure out and use for documentation. If the code is written like this: ```rust -#![doc(include = "docs/example.md")] +#![doc(include = "../docs/example.md")] fn my_func() { // Hidden implementation } @@ -100,8 +98,7 @@ fn my_func() { It should expand out to this at compile time: ```rust -//! # I'm an example -//! This is a markdown file that gets imported to Rust as a doc comment. +#![doc("# I'm an example\nThis is a markdown file that gets imported to Rust as a doc comment.")] fn my_func() { // Hidden implementation } @@ -111,7 +108,7 @@ In the case of this code: ```rust mod example { - #![doc(include = "docs/example.md")] + #![doc(include = "../docs/example.md")] fn my_func() { // Hidden implementation } @@ -122,15 +119,26 @@ It should expand out to: ```rust mod example { - //! # I'm an example - //! This is a markdown file that gets imported to Rust as a doc comment. + #![doc("# I'm an example\nThis is a markdown file that gets imported to Rust as a doc comment.")] fn my_func() { // Hidden implementation } } ``` -## Line numbers when errors occur +## Acceptable Paths + +If you've noticed the path given `../docs/example.md` is a relative path to +`src`. This was decided upon as a good first implementation and further RFCs +could be written to expand on what syntax is acceptable for paths. For instance +not being relative to `src`. + +## Missing Files or Incorrect Paths +If a file given to `include` is missing then this should trigger a compilation +as the given file was supposed to put into the code but for some reason or other +it is not there. + +## Line Numbers When Errors Occur As with all macros being expanded this brings up the question of line numbers and for documentation tests especially so, to keep things simple for the user the documentation should be treated separately from the code. Since the @@ -140,10 +148,10 @@ ignored by the compiler except for having the proper lines for error messages. For example if we have this: ```rust -#[doc(include = "docs/example.md")] // Line 1 -f my_func() { // Line 2 - // Hidden implementation // Line 3 -} // Line 4 +#[doc(include = "../docs/example.md")] // Line 1 +f my_func() { // Line 2 + // Hidden implementation // Line 3 +} // Line 4 ``` Then we would have a syntax error on line 2, however the doc comment comes @@ -209,9 +217,4 @@ comes to docs. # Unresolved questions [unresolved]: #unresolved-questions -- Should we include multiple items in one file or should each type/field be - allowed one file? If so how do we delimit it for the parser? -- Are there corner cases not covered in the current design? If there are what - would be a good design to solve them? -- Is this the syntax we want to use for the attribute? Do we want to use - something else? +- What would be best practices for adding docs to crates? From 1cefc60f773587002a6ce857a892d509988b4716 Mon Sep 17 00:00:00 2001 From: Michael Gattozzi Date: Tue, 19 Sep 2017 23:07:27 -0400 Subject: [PATCH 6/6] Typo fix --- text/0000-external-doc-attribute.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0000-external-doc-attribute.md b/text/0000-external-doc-attribute.md index e220c58de29..cb3d3834d4b 100644 --- a/text/0000-external-doc-attribute.md +++ b/text/0000-external-doc-attribute.md @@ -135,8 +135,8 @@ not being relative to `src`. ## Missing Files or Incorrect Paths If a file given to `include` is missing then this should trigger a compilation -as the given file was supposed to put into the code but for some reason or other -it is not there. +error as the given file was supposed to be put into the code but for some reason +or other it is not there. ## Line Numbers When Errors Occur As with all macros being expanded this brings up the question of line numbers