Skip to content

Commit

Permalink
Edit the "Compiler Source Code" chapter (rust-lang#1307)
Browse files Browse the repository at this point in the history
Co-authored-by: Noah Lev <[email protected]>
Co-authored-by: pierwill <[email protected]>
Co-authored-by: Yuki Okushi <[email protected]>
  • Loading branch information
4 people authored May 16, 2022
1 parent 8912c34 commit 2c54ffb
Showing 1 changed file with 25 additions and 23 deletions.
48 changes: 25 additions & 23 deletions src/doc/rustc-dev-guide/src/compiler-src.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,13 @@
<!-- toc -->

Now that we have [seen what the compiler does](./overview.md), let's take a
look at the structure of the contents of the rust-lang/rust repo.
look at the structure of the [`rust-lang/rust`] repository, where the rustc
source code lives.

[`rust-lang/rust`]: https://github.com/rust-lang/rust

> You may find it helpful to read the ["Overview of the compiler"](./overview.md)
> chapter, which introduces how the compiler works, before this one.
## Workspace structure

Expand All @@ -16,29 +22,17 @@ The repository consists of three main directories:

- `compiler/` contains the source code for `rustc`. It consists of many crates
that together make up the compiler.

- `library/` contains the standard libraries (`core`, `alloc`, `std`,
`proc_macro`, `test`), as well as the Rust runtime (`backtrace`, `rtstartup`,
`lang_start`).

- `src/` contains the source code for rustdoc, clippy, cargo, the build system,
language docs, etc.

## Standard library

The standard library crates are all in `library/`. They have intuitive names
like `std`, `core`, `alloc`, etc. There is also `proc_macro`, `test`, and
other runtime libraries.

This code is fairly similar to most other Rust crates except that it must be
built in a special way because it can use unstable features.
compiler tests, language docs, etc.

## Compiler

> You may find it helpful to read [The Overview Chapter](./overview.md) first,
> which gives an overview of how the compiler works. The crates mentioned in
> this section implement the compiler, and are underneath `compiler/`
The compiler is implemented in the various `compiler/` crates.
The `compiler/` crates all have names starting with `rustc_*`. These are a
collection of around 50 interdependent crates ranging in size from tiny to
huge. There is also the `rustc` crate which is the actual binary (i.e. the
Expand Down Expand Up @@ -87,7 +81,7 @@ explanation of these crates here.

### Big picture

The dependency structure is influenced strongly by two main factors:
The dependency structure is influenced by two main factors:

1. Organization. The compiler is a _huge_ codebase; it would be an impossibly
large crate. In part, the dependency structure reflects the code structure
Expand All @@ -101,12 +95,11 @@ At the very bottom of the dependency tree are a handful of crates that are used
by the whole compiler (e.g. [`rustc_span`]). The very early parts of the
compilation process (e.g. parsing and the AST) depend on only these.

Pretty soon after the AST is constructed, the compiler's [query system][query]
gets set up. The query system is set up in a clever way using function
After the AST is constructed and other early analysis is done, the compiler's [query system][query]
gets set up. The query system is set up in a clever way using function
pointers. This allows us to break dependencies between crates, allowing more
parallel compilation.

However, since the query system is defined in [`rustc_middle`], nearly all
The query system is defined in [`rustc_middle`], so nearly all
subsequent parts of the compiler depend on this crate. It is a really large
crate, leading to long compile times. Some efforts have been made to move stuff
out of it with limited success. Another unfortunate side effect is that sometimes
Expand All @@ -116,7 +109,7 @@ linting functionality is scattered across earlier parts of the crate,

[`rustc_lint`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_lint/index.html

More generally, in an ideal world, it seems like there would be fewer, more
Ideally there would be fewer, more
cohesive crates, with incremental and parallel compilation making sure compile
times stay reasonable. However, our incremental and parallel compilation haven't
gotten good enough for that yet, so breaking things into separate crates has
Expand Down Expand Up @@ -180,6 +173,15 @@ from `src/tools/`, such as [`tidy`] or [`compiletest`].

[bootstch]: ./building/bootstrapping.md

## Standard library

The standard library crates are all in `library/`. They have intuitive names
like `std`, `core`, `alloc`, etc. There is also `proc_macro`, `test`, and
other runtime libraries.

This code is fairly similar to most other Rust crates except that it must be
built in a special way because it can use unstable features.

## Other

There are a lot of other things in the `rust-lang/rust` repo that are related
Expand Down

0 comments on commit 2c54ffb

Please sign in to comment.