Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Changes at 6/20/2022 7:45:04 PM #6346

Closed
performanceautofiler bot opened this issue Jun 23, 2022 · 1 comment
Closed

[Perf] Changes at 6/20/2022 7:45:04 PM #6346

performanceautofiler bot opened this issue Jun 23, 2022 · 1 comment
Assignees
@EgorBo
Labels
arch-arm64 branch-refs/heads/main kind-micro os-windows perf-improvement runtime-coreclr untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Jun 23, 2022 •
edited
Loading

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 38794de5f2e3198b28df0915e82e33ab3d91f2ee
Compare 715c71ae6af5775402e54d3b8ab6c4a888da3cb5
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.WriteJson<BinaryData>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToStream - Duration of single invocation 1.46 μs 883.86 ns 0.61 0.01 False
SerializeToUtf8Bytes - Duration of single invocation 1.40 μs 817.43 ns 0.58 0.01 False
SerializeToString - Duration of single invocation 1.74 μs 1.14 μs 0.66 0.01 False
SerializeObjectProperty - Duration of single invocation 2.03 μs 1.38 μs 0.68 0.02 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;BinaryData&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeToStream

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 883.857165843224 < 1.3833268125103637.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 60.89621227893373 (T) = (0 -876.1515494432989) / Math.Sqrt((57969.32385303655 / (299)) + (158.7189346237011 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.5026936233069499 = (1761.794319368001 - 876.1515494432989) / 1761.794319368001 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;BinaryData&gt;.SerializeToUtf8Bytes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 817.4293255539347 < 1.333331239317457.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 65.80377891175843 (T) = (0 -829.5311740418587) / Math.Sqrt((46611.71141419696 / (299)) + (77.82109247689675 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.5036887498845598 = (1671.393049923637 - 829.5311740418587) / 1671.393049923637 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;BinaryData&gt;.SerializeToString

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1404520294081908 < 1.6518461566246312.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 66.86854793944667 (T) = (0 -1149.4868890126104) / Math.Sqrt((44686.97535549383 / (299)) + (99.24705718449978 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.42343251862332376 = (1993.6727722971273 - 1149.4868890126104) / 1993.6727722971273 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;BinaryData&gt;.SerializeObjectProperty

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3840413381175984 < 1.9004342344900051.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 50.54349435241192 (T) = (0 -1422.6919312919147) / Math.Sqrt((48337.569921029295 / (299)) + (1252.7299324804344 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.375699942022038 = (2278.859200974375 - 1422.6919312919147) / 2278.859200974375 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS Windows 10.0.19041
Baseline 38794de5f2e3198b28df0915e82e33ab3d91f2ee
Compare 715c71ae6af5775402e54d3b8ab6c4a888da3cb5
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Base64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteByteArrayAsBase64_NoEscaping - Duration of single invocation 163.83 ns 108.37 ns 0.66 0.01 False
WriteByteArrayAsBase64_HeavyEscaping - Duration of single invocation 1.05 μs 473.97 ns 0.45 0.01 False
WriteByteArrayAsBase64_HeavyEscaping - Duration of single invocation 162.82 ns 106.44 ns 0.65 0.01 False
WriteByteArrayAsBase64_NoEscaping - Duration of single invocation 1.04 μs 477.79 ns 0.46 0.01 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Base64*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_NoEscaping(NumberOfBytes: 100)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 108.36684464229948 < 155.07980627449516.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 63.98447900917435 (T) = (0 -108.58878728632578) / Math.Sqrt((492.90908746493767 / (299)) + (0.17792783854159552 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.4320187624165057 = (191.18375766833853 - 108.58878728632578) / 191.18375766833853 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_HeavyEscaping(NumberOfBytes: 1000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 473.9677445522669 < 0.9921985444759072.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 67.01556886135363 (T) = (0 -476.23833294212375) / Math.Sqrt((46121.384638755204 / (299)) + (7.577015542245413 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.6366262392136861 = (1310.6018770083433 - 476.23833294212375) / 1310.6018770083433 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_HeavyEscaping(NumberOfBytes: 100)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 106.43758991002588 < 154.8901704914402.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 61.88671612065644 (T) = (0 -108.34534500514994) / Math.Sqrt((489.5290681903971 / (299)) + (1.5322352856995118 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.4332050917283139 = (191.15440774781263 - 108.34534500514994) / 191.15440774781263 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_NoEscaping(NumberOfBytes: 1000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 477.78661996353884 < 0.9924306789724591.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 66.93502045736213 (T) = (0 -477.48554261895856) / Math.Sqrt((46013.804942921975 / (299)) + (8.9571348318835 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.6356511559406868 = (1310.5175174954788 - 477.48554261895856) / 1310.5175174954788 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 38794de5f2e3198b28df0915e82e33ab3d91f2ee
Compare 715c71ae6af5775402e54d3b8ab6c4a888da3cb5
Diff Diff

Improvements in System.Buffers.Text.Tests.Base64Tests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Base64Encode - Duration of single invocation 881.56 ns 314.21 ns 0.36 0.01 False
Base64EncodeDestinationTooSmall - Duration of single invocation 878.57 ns 320.48 ns 0.36 0.01 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Base64Tests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Base64Tests.Base64Encode(NumberOfBytes: 1000)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 314.2116356221086 < 837.5219958614132.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 66.69610829381433 (T) = (0 -316.6646326220751) / Math.Sqrt((46494.290363698245 / (299)) + (1.0559457142302267 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.7243217073293606 = (1148.6745276691163 - 316.6646326220751) / 1148.6745276691163 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Base64Tests.Base64EncodeDestinationTooSmall(NumberOfBytes: 1000)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 320.48052744398865 < 834.7421562527652.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 66.17870934756729 (T) = (0 -321.4121190572265) / Math.Sqrt((46408.29315684704 / (299)) + (0.44887072290911806 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.7195386182162227 = (1146.0120356428263 - 321.4121190572265) / 1146.0120356428263 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 38794de5f2e3198b28df0915e82e33ab3d91f2ee
Compare 715c71ae6af5775402e54d3b8ab6c4a888da3cb5
Diff Diff

Improvements in System.Security.Cryptography.Primitives.Tests.Performance.Perf_CryptoStream

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Base64EncodeAsync - Duration of single invocation 10.41 ms 4.34 ms 0.42 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Security.Cryptography.Primitives.Tests.Performance.Perf_CryptoStream*'

Payloads

Baseline
Compare

Histogram

System.Security.Cryptography.Primitives.Tests.Performance.Perf_CryptoStream.Base64EncodeAsync

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.337096774193549 < 9.889730940476188.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 69.88888112518408 (T) = (0 -4345378.305785971) / Math.Sqrt((4679257342344.113 / (299)) + (129658770.29792878 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.6680892681113276 = (13092009.050323427 - 4345378.305785971) / 13092009.050323427 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 38794de5f2e3198b28df0915e82e33ab3d91f2ee
Compare 715c71ae6af5775402e54d3b8ab6c4a888da3cb5
Diff Diff

Improvements in System.Text.Json.Reader.Tests.Perf_Base64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadBase64EncodedByteArray_NoEscaping - Duration of single invocation 1.07 μs 885.39 ns 0.82 0.02 False
ReadBase64EncodedByteArray_HeavyEscaping - Duration of single invocation 178.11 ns 143.17 ns 0.80 0.15 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Reader.Tests.Perf_Base64*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_NoEscaping(NumberOfBytes: 1000)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 885.3865158963844 < 1.0188946561235308.
IsChangePoint: Marked as a change because one of 5/17/2022 8:30:53 PM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 52.390398702396496 (T) = (0 -884.9523223220303) / Math.Sqrt((11067.881540533906 / (299)) + (9.07081032997818 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.26717047542538264 = (1207.5827906029238 - 884.9523223220303) / 1207.5827906029238 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_HeavyEscaping(NumberOfBytes: 100)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 143.168082634513 < 167.25128424017848.
IsChangePoint: Marked as a change because one of 5/17/2022 8:30:53 PM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 50.998669826721276 (T) = (0 -143.66755753544726) / Math.Sqrt((166.1085303069004 / (299)) + (0.7188910647418901 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.2205551645447258 = (184.3203662405832 - 143.66755753544726) / 184.3203662405832 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 38794de5f2e3198b28df0915e82e33ab3d91f2ee
Compare 715c71ae6af5775402e54d3b8ab6c4a888da3cb5
Diff Diff

Improvements in System.IO.Tests.Perf_FileStream

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LockUnlock - Duration of single invocation 42.25 μs 36.83 μs 0.87 0.47 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_FileStream*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_FileStream.LockUnlock(fileSize: 1024, options: Asynchronous)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.83300827423168 < 40.146437191150035.
IsChangePoint: Marked as a change because one of 3/2/2022 12:52:37 PM, 3/3/2022 4:26:18 PM, 3/9/2022 10:56:44 AM, 3/11/2022 10:05:40 PM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 4.080860358420332 (T) = (0 -36008.077403711715) / Math.Sqrt((5378018.658824573 / (299)) + (14618179.482487706 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.1268160912001915 = (41237.67861595712 - 36008.077403711715) / 41237.67861595712 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 38794de5f2e3198b28df0915e82e33ab3d91f2ee
Compare 715c71ae6af5775402e54d3b8ab6c4a888da3cb5
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<BinaryData>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromString - Duration of single invocation 1.56 μs 1.38 μs 0.89 0.01 False
DeserializeFromUtf8Bytes - Duration of single invocation 1.32 μs 1.12 μs 0.85 0.01 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;BinaryData&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<BinaryData>.DeserializeFromString

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3817041803004764 < 1.477783394712726.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 40.509887385569364 (T) = (0 -1369.4763464547552) / Math.Sqrt((12562.29381232156 / (299)) + (242.42428563675523 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.1940558143533046 = (1699.2198353734354 - 1369.4763464547552) / 1699.2198353734354 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;BinaryData&gt;.DeserializeFromUtf8Bytes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1200287666307083 < 1.257796936502981.
IsChangePoint: Marked as a change because one of 5/17/2022 8:30:53 PM, 6/20/2022 12:28:47 PM, 6/23/2022 3:07:42 AM falls between 6/14/2022 10:05:19 AM and 6/23/2022 3:07:42 AM.
IsImprovementStdDev: Marked as improvement because 39.167370090437934 (T) = (0 -1132.7713449719854) / Math.Sqrt((11707.154134651015 / (299)) + (331.2281069644607 / (10))) is greater than 1.9677212881555053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (10) - 2, .975) and 0.2271763582606389 = (1465.7565889450605 - 1132.7713449719854) / 1465.7565889450605 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@EgorBo EgorBo

Labels
arch-arm64 branch-refs/heads/main kind-micro os-windows perf-improvement runtime-coreclr untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@EgorBo @kunalspathak