Replay Binary Log Tool #72619
Merged
Replay Binary Log Tool #72619
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This adds a tool for replaying compilation events from binary logs directly to the compiler server. This is an effective tool for profiling the compiler server as it removes MSBuild overhead from the logs leaving just the compiler. Further because it's integrated into our code base it makes it very easy to evaluate the efficacy of performance fixes. One simply needs to run `replay` with and without the changes against the same binary log and evaluate the differences in perfview.
dotnet-issue-labeler
bot
added
Area-Infrastructure
untriaged
|
@dotnet/roslyn-compiler PTAL |
jaredpar
commented
Mar 20, 2024
jaredpar
commented
Mar 20, 2024
Comment on lines
+10
to
+19
| <Compile Include="..\..\Compilers\Core\CommandLine\BuildProtocol.cs" /> | ||
| <Compile Include="..\..\Compilers\Core\CommandLine\CompilerServerLogger.cs" /> | ||
| <Compile Include="..\..\Compilers\Core\CommandLine\NativeMethods.cs" /> | ||
| <Compile Include="..\..\Compilers\Core\Portable\CommitHashAttribute.cs" /> | ||
| <Compile Include="..\..\Compilers\Core\Portable\InternalUtilities\PlatformInformation.cs" /> | ||
| <Compile Include="..\..\Compilers\Core\Portable\InternalUtilities\Debug.cs" /> | ||
| <Compile Include="..\..\Compilers\Core\Portable\InternalUtilities\NullableAttributes.cs" /> | ||
| <Compile Include="..\..\Compilers\Shared\BuildServerConnection.cs" /> | ||
| <Compile Include="..\..\Compilers\Shared\NamedPipeUtil.cs" /> | ||
| <Compile Include="..\..\Compilers\Shared\RuntimeHostInfo.cs" /> |
There was a problem hiding this comment.
It bothers me a bit that all of these files are needed to effectively run the client side of the build server. Considering changing this in a future PR to be a lot simpler. Essentially:
- Create a shared project that has all of this logic in it.
- Reference that shared project in MS.CA. Then vbc, csc and VBCSCompiler can just use the logic as they have IVT to MS.CA.
- Reference that shared project in MS.Build.Tasks.CA.csproj. It does not reference MS.CA (by very explicit design) hence needs its own copy of the source.
- Reference that project in MS.CA as well.
jaredpar
commented
Mar 20, 2024
| "${workspaceFolder}/scripts/vscode-build.ps1", | ||
| "-filePath", | ||
| "${file}", | ||
| "-msbuildEngine", | ||
| "dotnet" | ||
| ], | ||
| "windows": { | ||
| "command": "powershell", |
There was a problem hiding this comment.
There is no need for this anymore as part of our repo setup includes installing pwsh as a local tool.
333fred
reviewed
Mar 20, 2024
Co-authored-by: Fred Silberberg <[email protected]>
333fred
reviewed
Mar 20, 2024
333fred
reviewed
Mar 20, 2024
Co-authored-by: Fred Silberberg <[email protected]>
333fred
approved these changes
Mar 21, 2024
|
@RikkiGibson, @jjonescz PTAL |
jjonescz
approved these changes
Mar 21, 2024
src/Tools/Replay/Replay.cs
Outdated
| var stopwatch = new Stopwatch(); | ||
| stopwatch.Start(); | ||
|
|
||
| var compilerCalls = ReadAllCompilerCalls(options.BinlogPath); |
There was a problem hiding this comment.
Should reading compiler calls be outside the stopwatch?
There was a problem hiding this comment.
Good catch. Yeah that call should be fast compared to compilation times but the point is to measure compilation time so should move this out.
Co-authored-by: Jan Jones <[email protected]>
RikkiGibson
reviewed
Mar 21, 2024
|
|
||
| var options = new Mono.Options.OptionSet() | ||
| { | ||
| { "b|binlogPath=", "The binary log to relpay", v => binlogPath = v }, |
There was a problem hiding this comment.
Suggested change
| { "b|binlogPath=", "The binary log to relpay", v => binlogPath = v }, | |
| { "b|binlogPath=", "The binary log to replay", v => binlogPath = v }, |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds a tool for replaying compilation events from binary logs directly to the compiler server. This is an effective tool for profiling the compiler server as it removes MSBuild overhead from the logs leaving just the compiler.
Further because it's integrated into our code base it makes it very easy to evaluate the efficacy of performance fixes. One simply needs to run
replaywith and without the changes against the same binary log and evaluate the differences in perfview.