Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates in response to source-build-internal team renamed to source-build #15148

Merged
merged 3 commits into from
Oct 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Users referenced in this file will automatically be requested as reviewers for PRs that modify the given paths.
# See https://help.github.com/articles/about-code-owners/

/eng/SourceBuild* @dotnet/source-build-internal
/eng/SourceBuild* @dotnet/source-build
37 changes: 19 additions & 18 deletions Documentation/UnifiedBuild/VMR-Permissible-Sources.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,31 +6,30 @@ This document provides guidelines on the types of source code that are permissib

## Binaries

### Policy
### Binary Policy
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There were a number of markdown linter violations in this file I decided to address since touching.


OSS licensed binaries are allowed in the VMR, but the inclusion status of all binaries that could be included in the VMR must be made explicit. Any binary not explicitly specified is considered "new" and is not allowed in the VMR for all build scenarios. In general, it is preferred to not have binaries in the VMR, but they can be included as long as they are not required to build the source-build product.

To allow a binary into the VMR, you must add the binary or its file glob pattern to either [`allowed-sb-binaries.txt`](https://github.com/dotnet/dotnet/blob/main/src/installer/src/VirtualMonoRepo/allowed-sb-binaries.txt) or [`allowed-vmr-binaries.txt`](https://github.com/dotnet/dotnet/blob/main/src/installer/src/VirtualMonoRepo/allowed-vmr-binaries.txt). Which file to add the binary or its pattern to depends on whether or not the binary is allowed for source build. For help determining if the newly detected binary is allowed in the source-build context, please contact a member of the source-build team at [@dotnet/source-build-internal](https://github.com/orgs/dotnet/teams/source-build-internal).
To allow a binary into the VMR, you must add the binary or its file glob pattern to either [`allowed-sb-binaries.txt`](https://github.com/dotnet/dotnet/blob/main/src/installer/src/VirtualMonoRepo/allowed-sb-binaries.txt) or [`allowed-vmr-binaries.txt`](https://github.com/dotnet/dotnet/blob/main/src/installer/src/VirtualMonoRepo/allowed-vmr-binaries.txt). Which file to add the binary or its pattern to depends on whether or not the binary is allowed for source build. For help determining if the newly detected binary is allowed in the source-build context, please contact a member of the source-build team at [@dotnet/source-build](https://github.com/orgs/dotnet/teams/source-build).

If the binary is allowed for source-build, add it to `allowed-sb-binaries.txt`. This binary is now allowed in the VMR for source-build and non-source-build scenarios. The binary will not be removed during a build of the source-build product.

If the binary is not allowed for source-build, add it to `allowed-vmr-binaries.txt`. This binary is now allowed in the VMR for non-source-build scenarios and will be removed during a build of the source-build product.

> [!NOTE]
> `allowed-sb-binaries.txt` is a subset of `allowed-vmr-binaries.txt`, so is imported at the top of `allowed-vmr-binaries.txt`. This means that all binaries & exclusions in `allowed-sb-binaries.txt` are also included in `allowed-vmr-binaries.txt`.
> [!NOTE]
>
> - `allowed-sb-binaries.txt` is a subset of `allowed-vmr-binaries.txt`, so is imported at the top of `allowed-vmr-binaries.txt`. This means that all binaries & exclusions in `allowed-sb-binaries.txt` are also included in `allowed-vmr-binaries.txt`.
> - It is best to target a single file or use a specific pattern when adding to either `allowed-sb-binaries.txt` or `allowed-vmr-binaries.txt`. Otherwise, vague patterns may permit binaries into the VMR that were previously undetected. For example, avoid patterns such as `**/test/**` and instead use a more specific patterns like `src/arcade/test/**/*.png`.

> [!IMPORTANT]
> It is best to target a single file or use a specific pattern when adding to either `allowed-sb-binaries.txt` or `allowed-vmr-binaries.txt`. Otherwise, vague patterns may permit binaries into the VMR that were previously undetected. For example, avoid patterns such as `**/test/**` and instead use a more specific patterns like `src/arcade/test/**/*.png`.

When adding a binary or pattern to either `allowed-sb-binaries.txt` or `allowed-vmr-binaries.txt`, remember to include a link to the relevant issue and tag [@dotnet/source-build-internal](https://github.com/orgs/dotnet/teams/source-build-internal) as a reviewer.
When adding a binary or pattern to either `allowed-sb-binaries.txt` or `allowed-vmr-binaries.txt`, remember to include a link to the relevant issue and tag [@dotnet/source-build](https://github.com/orgs/dotnet/teams/source-build) as a reviewer.

### Validation and Cleaning

The [BinaryTool](https://github.com/dotnet/dotnet/tree/main/eng/tools/BinaryToolKit) is used to validate binaries in the VMR and clean binaries from the VMR. You can run the tool locally by running `./eng/detect-binaries.sh` to detect binaries and `./eng/detect-binaries.sh --clean` to remove binaries. This functionality is also exposed via `./prep-source-build/sh`.

#### Validation

The tool flags all binaries not listed in `allowed-vmr-binaries.txt` and `allowed-sb-binaries.txt`. Note that the tool only uses `allowed-vmr-binaries.txt` as a baseline during validation, but, as noted above, `allowed-sb-binaries.txt` is imported at the top of `allowed-vmr-binaries.txt`, meaning that all binaries in `allowed-sb-binaries.txt` are also relevant.
The tool flags all binaries not listed in `allowed-vmr-binaries.txt` and `allowed-sb-binaries.txt`. Note that the tool only uses `allowed-vmr-binaries.txt` as a baseline during validation, but, as noted above, `allowed-sb-binaries.txt` is imported at the top of `allowed-vmr-binaries.txt`, meaning that all binaries in `allowed-sb-binaries.txt` are also relevant.

To run default validation, execute `eng/detect-binaries.sh`.

Expand All @@ -48,14 +47,15 @@ An example output is as follows:
```

In this example, the tool detected 2 new binaries:
- `src/wpf/src/Microsoft.DotNet.Wpf/src/Shared/Tracing/resources/MSG00001.bin`
- `src/wpf/src/Microsoft.DotNet.Wpf/src/Shared/Tracing/resources/wpf-etwTEMP.BIN`.

- `src/wpf/src/Microsoft.DotNet.Wpf/src/Shared/Tracing/resources/MSG00001.bin`
- `src/wpf/src/Microsoft.DotNet.Wpf/src/Shared/Tracing/resources/wpf-etwTEMP.BIN`.

If these binaries are permitted for source-build, add the binaries and/or relevent file glob pattern(s), such as `src/wpf/src/Microsoft.DotNet.Wpf/src/Shared/Tracing/resources/*.bin`, to `allowed-sb-binaries.txt`. Otherwise, add the binaries or relevent file glob pattern(s) to `allowed-vmr-binaries.txt`.

#### Cleaning

The tool removes all binaries from the VMR that are not listed in `allowed-vmr-binaries.txt` and `allowed-sb-binaries.txt`. Again, note that the tool only uses `allowed-vmr-binaries.txt` as a baseline during validation, but, as noted above, `allowed-sb-binaries.txt` is imported at the top of `allowed-vmr-binaries.txt`, meaning that all binaries in `allowed-sb-binaries.txt` are also relevant.
The tool removes all binaries from the VMR that are not listed in `allowed-vmr-binaries.txt` and `allowed-sb-binaries.txt`. Again, note that the tool only uses `allowed-vmr-binaries.txt` as a baseline during validation, but, as noted above, `allowed-sb-binaries.txt` is imported at the top of `allowed-vmr-binaries.txt`, meaning that all binaries in `allowed-sb-binaries.txt` are also relevant.

To run cleaning, execute `./eng/detect-binaries.sh --clean` or `./prep-source-build.sh`. Executing `./prep-source-build.sh` will build the tool using previously source-built artifacts and remove non-source-build allowed binaries from the VMR whereas executing `./eng/detect-binaries.sh --clean` will build the tool using online resources and remove new binaries from the VMR.

Expand All @@ -72,15 +72,16 @@ An example output is as follows:
16:42:24 info: BinaryTool[0] Finished all binary tasks. Took 4.9003778 seconds.
```

In this example, the tool removed 2 binaries:
- `src/wpf/src/Microsoft.DotNet.Wpf/src/Shared/Tracing/resources/MSG00001.bin`
- `src/wpf/src/Microsoft.DotNet.Wpf/src/Shared/Tracing/resources/wpf-etwTEMP.BIN`.

In this example, the tool removed 2 binaries:

- `src/wpf/src/Microsoft.DotNet.Wpf/src/Shared/Tracing/resources/MSG00001.bin`
- `src/wpf/src/Microsoft.DotNet.Wpf/src/Shared/Tracing/resources/wpf-etwTEMP.BIN`.

If these binaries are permitted for source-build, add the binaries and/or relevent file glob pattern(s), such as `src/wpf/src/Microsoft.DotNet.Wpf/src/Shared/Tracing/resources/*.bin`, to `allowed-sb-binaries.txt`. Otherwise, add the binaries or relevent file glob pattern(s) to `allowed-vmr-binaries.txt`.

## Licenses

### Policy
### License Policy

The VMR does not permit code and binaries licensed under non-OSS licenses. For a list of approved OSS licenses, you can check out the [OSI-approved list of licenses](https://opensource.org/licenses/alphabetical).

Expand All @@ -92,4 +93,4 @@ Licenses are detected by the [license scan test](https://github.com/dotnet/dotne

Common cases for adding a license to [`LicenseExclusions.txt`](https://github.com/dotnet/dotnet/blob/main/test/Microsoft.DotNet.SourceBuild.SmokeTests/assets/LicenseExclusions.txt) include false positives, licenses related to test data, or needing to get a clean scan result with a relevant backport issue to remove the offending license later.

To have a license be added to the list of exclusions in [`LicenseExclusions.txt`](https://github.com/dotnet/dotnet/blob/main/test/Microsoft.DotNet.SourceBuild.SmokeTests/assets/LicenseExclusions.txt), open a PR, include a link to the relevant issue above the exclusion, and tag [@dotnet/source-build-internal](https://github.com/orgs/dotnet/teams/source-build-internal) as a reviewer.
To have a license be added to the list of exclusions in [`LicenseExclusions.txt`](https://github.com/dotnet/dotnet/blob/main/test/Microsoft.DotNet.SourceBuild.SmokeTests/assets/LicenseExclusions.txt), open a PR, include a link to the relevant issue above the exclusion, and tag [@dotnet/source-build](https://github.com/orgs/dotnet/teams/source-build) as a reviewer.
2 changes: 1 addition & 1 deletion eng/SourceBuild.props → eng/DotNetBuild.props
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
<!-- Whenever altering this or other Source Build files, please include @dotnet/source-build-internal as a reviewer. -->
<!-- When altering this file, please include @dotnet/product-construction as a reviewer. -->

<Project>

Expand Down
2 changes: 1 addition & 1 deletion eng/SourceBuildPrebuiltBaseline.xml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
<!-- Whenever altering this or other Source Build files, please include @dotnet/source-build-internal as a reviewer. -->
<!-- When altering this file or making other Source Build related changes, include @dotnet/source-build as a reviewer. -->
<!-- See https://aka.ms/dotnet/prebuilts for guidance on what pre-builts are and how to eliminate them. -->

<UsageData>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,8 @@ public class ValidateUsageAgainstBaseline : Microsoft.Build.Utilities.Task
private readonly string _preBuiltDocMessage = "See https://aka.ms/dotnet/prebuilts " +
"for guidance on what pre-builts are and how to eliminate them.";

private readonly string _reviewRequestMessage = "Whenever altering this " +
"or other Source Build files, please include @dotnet/source-build-internal " +
"as a reviewer.";
private readonly string _reviewRequestMessage = "When altering this file or making " +
"other Source Build related changes, include @dotnet/source-build as a reviewer.";

public override bool Execute()
{
Expand Down
Loading