[Perf] Windows/arm64: 120 Improvements on 4/26/2023 12:55:36 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in Exceptions.Handling

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ThrowAndCatchDeep - Duration of single invocation	24.98 μs	22.33 μs	0.89	0.10	False
ThrowAndCatch_ManyCatchBlocks - Duration of single invocation	8.34 μs	7.43 μs	0.89	0.10	False
ThrowAndCatch - Duration of single invocation	8.13 μs	7.17 μs	0.88	0.05	False
CatchAndRethrowDeep - Duration of single invocation	109.70 μs	104.18 μs	0.95	0.06	False
ThrowAndCatchWhen - Duration of single invocation	10.63 μs	9.72 μs	0.91	0.06	False
CatchAndThrowOtherDeep - Duration of single invocation	111.84 μs	99.40 μs	0.89	0.04	False
ThrowAndCatch_ManyCatchBlocks - Duration of single invocation	10.57 μs	9.81 μs	0.93	0.11	False
ThrowAndCatchDeepRecursive - Duration of single invocation	22.77 μs	20.55 μs	0.90	0.10	False
ThrowAndCatch - Duration of single invocation	23.32 μs	21.44 μs	0.92	0.08	False
MultipleNestedTryCatch_LastCatches - Duration of single invocation	10.43 μs	9.78 μs	0.94	0.07	False
ThrowAndCatchFinally - Duration of single invocation	7.80 μs	7.09 μs	0.91	0.08	False
ThrowAndCatchDeepRecursive - Duration of single invocation	26.16 μs	23.15 μs	0.89	0.09	False
ThrowAndCatchDeep - Duration of single invocation	21.46 μs	19.42 μs	0.91	0.08	False
TryAndFinallyDeep - Duration of single invocation	22.40 μs	20.26 μs	0.90	0.09	False
TryAndFinallyDeep - Duration of single invocation	25.72 μs	23.07 μs	0.90	0.07	False
ThrowAndCatch - Duration of single invocation	26.55 μs	24.58 μs	0.93	0.04	False
ThrowAndCatchDeepRecursive - Duration of single invocation	42.02 μs	37.75 μs	0.90	0.07	False
ThrowAndCatchDeepRecursive - Duration of single invocation	39.06 μs	35.31 μs	0.90	0.07	False
TryAndCatchDeep_CaugtAtTheTop - Duration of single invocation	26.00 μs	23.31 μs	0.90	0.12	False
MultipleNestedTryCatch_LastCatches - Duration of single invocation	8.18 μs	7.25 μs	0.89	0.06	False
ThrowAndCatch_ManyCatchBlocks - Duration of single invocation	23.61 μs	22.05 μs	0.93	0.10	False
ThrowAndCatchWhenFinally - Duration of single invocation	10.46 μs	9.67 μs	0.92	0.09	False
ThrowAndCatch_ManyCatchBlocks - Duration of single invocation	26.71 μs	24.56 μs	0.92	0.08	False
ThrowAndCatchDeep - Duration of single invocation	37.37 μs	33.86 μs	0.91	0.08	False
ThrowAndCatchDeep - Duration of single invocation	40.10 μs	36.92 μs	0.92	0.09	False
TryAndCatchDeep_CaugtAtTheTop - Duration of single invocation	23.55 μs	20.37 μs	0.86	0.11	False
CatchAndThrowOtherDeep - Duration of single invocation	115.39 μs	101.98 μs	0.88	0.06	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Exceptions.Handling*'

Payloads

Baseline
Compare

Histogram

Exceptions.Handling.ThrowAndCatchDeep(kind: Hardware)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 3:00:37 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 46.22607075574214 (T) = (0 -22333.277412496227) / Math.Sqrt((603787.6245226247 / (299)) + (3917.474768753568 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10662871771154657 = (24998.87544547824 - 22333.277412496227) / 24998.87544547824 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

Exceptions.Handling.ThrowAndCatch_ManyCatchBlocks(kind: Software)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 15.547566072896746 (T) = (0 -7353.122036016229) / Math.Sqrt((63308.44497188426 / (299)) + (4871.118949325734 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08306171059241127 = (8019.211457258372 - 7353.122036016229) / 8019.211457258372 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

Exceptions.Handling.ThrowAndCatch(kind: Software)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/5/2023 11:54:18 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 26.476133566412425 (T) = (0 -7161.490614483999) / Math.Sqrt((58012.809911942946 / (299)) + (942.7605341549429 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.07693653490690944 = (7758.394612402697 - 7161.490614483999) / 7758.394612402697 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Exceptions.Handling.CatchAndRethrowDeep(kind: Hardware)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/18/2023 1:15:50 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 8.48416390671005 (T) = (0 -102119.41593013467) / Math.Sqrt((8994796.861177586 / (299)) + (3615957.169015968 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08453679337725889 = (111549.44861942189 - 102119.41593013467) / 111549.44861942189 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Exceptions.Handling.ThrowAndCatchWhen(kind: Hardware)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 42.05078628744065 (T) = (0 -9683.819644034542) / Math.Sqrt((81838.2995168058 / (299)) + (1154.1873073020265 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10025448982933033 = (10762.842975673922 - 9683.819644034542) / 10762.842975673922 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Exceptions.Handling.CatchAndThrowOtherDeep(kind: Software)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/9/2023 2:25:45 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 21.24960355071584 (T) = (0 -99387.88253901557) / Math.Sqrt((9168082.4779317 / (299)) + (299609.85462618625 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.07170702076448639 = (107065.2097583087 - 99387.88253901557) / 107065.2097583087 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

Exceptions.Handling.ThrowAndCatch_ManyCatchBlocks(kind: Hardware)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 4/17/2023 5:51:51 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 54.292466005260906 (T) = (0 -9795.313392396092) / Math.Sqrt((83520.63960689714 / (299)) + (384.6855978866435 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10063601843929436 = (10891.378344280429 - 9795.313392396092) / 10891.378344280429 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Exceptions.Handling.ThrowAndCatchDeepRecursive(kind: Software)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/4/2023 9:40:21 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 42.51541954160579 (T) = (0 -20564.340185416546) / Math.Sqrt((613495.8349725878 / (299)) + (4216.118639748355 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10838545089103598 = (23064.15951373555 - 20564.340185416546) / 23064.15951373555 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Exceptions.Handling.ThrowAndCatch(kind: ReflectionSoftware)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 7:22:28 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 12.313361594456163 (T) = (0 -21276.603609373782) / Math.Sqrt((527827.7828530897 / (299)) + (81498.90547073298 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08961594528154948 = (23371.019625287572 - 21276.603609373782) / 23371.019625287572 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Exceptions.Handling.MultipleNestedTryCatch_LastCatches(kind: Hardware)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 57.322303475620124 (T) = (0 -9797.469454690947) / Math.Sqrt((77157.85523515711 / (299)) + (201.84460763856111 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09545625526361738 = (10831.393740439044 - 9797.469454690947) / 10831.393740439044 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Exceptions.Handling.ThrowAndCatchFinally(kind: Software)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/5/2023 11:54:18 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 12.828768112312375 (T) = (0 -7124.711823110684) / Math.Sqrt((64209.08567488885 / (299)) + (7127.441258228253 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08395229254251106 = (7777.664596623993 - 7124.711823110684) / 7777.664596623993 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Exceptions.Handling.ThrowAndCatchDeepRecursive(kind: Hardware)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/2/2023 4:26:55 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 11.487769550215042 (T) = (0 -23478.95282100026) / Math.Sqrt((646213.6926931262 / (299)) + (186695.60338314128 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.11044578347265646 = (26394.066134223707 - 23478.95282100026) / 26394.066134223707 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

Exceptions.Handling.ThrowAndCatchDeep(kind: Software)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/1/2023 10:50:04 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 35.693067892546715 (T) = (0 -19409.376638325437) / Math.Sqrt((517940.7044715429 / (299)) + (5688.819764798269 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09972665004500697 = (21559.425966897456 - 19409.376638325437) / 21559.425966897456 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Exceptions.Handling.TryAndFinallyDeep(kind: Software)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 7:22:28 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 32.766309762923115 (T) = (0 -20198.158360165417) / Math.Sqrt((565765.4788141109 / (299)) + (9652.28275646285 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1039109304351346 = (22540.346764829417 - 20198.158360165417) / 22540.346764829417 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Exceptions.Handling.TryAndFinallyDeep(kind: Hardware)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/9/2023 3:58:44 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 37.78893414768731 (T) = (0 -23037.66784496076) / Math.Sqrt((669084.4477439505 / (299)) + (10867.239799466412 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1115600544848197 = (25930.472803766035 - 23037.66784496076) / 25930.472803766035 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Exceptions.Handling.ThrowAndCatch(kind: ReflectionHardware)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/17/2023 6:13:48 AM, 3/10/2023 12:53:36 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 28.444956383679855 (T) = (0 -24446.714448900555) / Math.Sqrt((555381.692346465 / (299)) + (13959.008153914008 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08582614853351847 = (26741.86579465613 - 24446.714448900555) / 26741.86579465613 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Exceptions.Handling.ThrowAndCatchDeepRecursive(kind: ReflectionHardware)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/27/2023 12:02:42 PM, 4/25/2023 1:02:54 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 20.848680779521114 (T) = (0 -38014.2628563658) / Math.Sqrt((1869311.5003869017 / (299)) + (116184.1815787877 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10419672753011716 = (42435.949973205585 - 38014.2628563658) / 42435.949973205585 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Exceptions.Handling.ThrowAndCatchDeepRecursive(kind: ReflectionSoftware)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/5/2023 11:54:18 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 15.653880781901684 (T) = (0 -35301.374582180804) / Math.Sqrt((1729327.7429705132 / (299)) + (142475.10083175247 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09284825115793871 = (38914.51967902992 - 35301.374582180804) / 38914.51967902992 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Exceptions.Handling.TryAndCatchDeep_CaugtAtTheTop(kind: Hardware)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 51.7518400273622 (T) = (0 -23244.800581503103) / Math.Sqrt((766612.3415190781 / (299)) + (2947.540645270379 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.11706442909290743 = (26326.723429686186 - 23244.800581503103) / 26326.723429686186 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Exceptions.Handling.MultipleNestedTryCatch_LastCatches(kind: Software)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 28.01128222979606 (T) = (0 -7266.877590635417) / Math.Sqrt((58810.5164052213 / (299)) + (1136.6424842838635 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08464911073199037 = (7938.898269325563 - 7266.877590635417) / 7938.898269325563 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Exceptions.Handling.ThrowAndCatch_ManyCatchBlocks(kind: ReflectionSoftware)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/20/2023 3:29:46 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 7.687503280401161 (T) = (0 -21914.07098446437) / Math.Sqrt((629166.1516948566 / (299)) + (162356.41239643202 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.07679248775055464 = (23736.885471251793 - 21914.07098446437) / 23736.885471251793 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Exceptions.Handling.ThrowAndCatchWhenFinally(kind: Hardware)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 4/17/2023 2:42:45 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 23.715695380441776 (T) = (0 -9731.413086648836) / Math.Sqrt((95529.09915526418 / (299)) + (5107.872121924842 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09876465765375445 = (10797.860036552056 - 9731.413086648836) / 10797.860036552056 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

Exceptions.Handling.ThrowAndCatch_ManyCatchBlocks(kind: ReflectionHardware)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 25.178346617535357 (T) = (0 -24678.12408427183) / Math.Sqrt((758169.0024773375 / (299)) + (21343.322917116755 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09109603844074689 = (27151.519993306814 - 24678.12408427183) / 27151.519993306814 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Exceptions.Handling.ThrowAndCatchDeep(kind: ReflectionSoftware)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/5/2023 11:54:18 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 42.235306120342024 (T) = (0 -33790.6046822056) / Math.Sqrt((1719564.308427593 / (299)) + (5431.558199236315 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09803382330839246 = (37463.27252109254 - 33790.6046822056) / 37463.27252109254 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline

Exceptions.Handling.ThrowAndCatchDeep(kind: ReflectionHardware)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/4/2023 3:36:11 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 45.224012781388666 (T) = (0 -36815.930862132205) / Math.Sqrt((1789653.4725981758 / (299)) + (7463.921438298135 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10158721065814608 = (40978.858826244315 - 36815.930862132205) / 40978.858826244315 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Exceptions.Handling.TryAndCatchDeep_CaugtAtTheTop(kind: Software)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 17.548751290486294 (T) = (0 -20141.656120238298) / Math.Sqrt((713404.3349439885 / (299)) + (52407.6219000806 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10934465976389432 = (22614.422448642035 - 20141.656120238298) / 22614.422448642035 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

Exceptions.Handling.CatchAndThrowOtherDeep(kind: Hardware)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/10/2023 12:53:36 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 3.1724208115835015 (T) = (0 -104728.13460685754) / Math.Sqrt((9154621.049660238 / (299)) + (13485693.77329475 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.060541743847139354 = (111477.15603216442 - 104728.13460685754) / 111477.15603216442 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.TryGetValueTrue<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedList - Duration of single invocation	30.33 μs	28.68 μs	0.95	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueTrue<Int32, Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueTrue<Int32, Int32>.SortedList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 8:11:05 PM, 1/18/2023 3:44:12 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 85.58925007167839 (T) = (0 -28670.11401638344) / Math.Sqrt((30686.041543694024 / (299)) + (988.6821171100407 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.0584355344755099 = (30449.443523139977 - 28670.11401638344) / 30449.443523139977 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CtorGivenSize<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	697.01 ns	630.69 ns	0.90	0.05	False
SortedList - Duration of single invocation	271.24 ns	250.72 ns	0.92	0.06	False
Array - Duration of single invocation	125.51 ns	115.08 ns	0.92	0.03	False
List - Duration of single invocation	137.14 ns	128.36 ns	0.94	0.05	False
Queue - Duration of single invocation	139.07 ns	126.81 ns	0.91	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorGivenSize<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorGivenSize<Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 29.824022043850597 (T) = (0 -628.8201970127387) / Math.Sqrt((407.1357368996259 / (299)) + (2.8075502326242785 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.06706841570632785 = (674.0260567861706 - 628.8201970127387) / 674.0260567861706 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorGivenSize<Int32>.SortedList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 15.053804869058123 (T) = (0 -251.0090350858086) / Math.Sqrt((55.99308019513914 / (299)) + (3.408213400227195 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.06453843443969871 = (268.326400920026 - 251.0090350858086) / 268.326400920026 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorGivenSize<Int32>.Array(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 22.195609752987494 (T) = (0 -114.77270899889635) / Math.Sqrt((9.872742403182041 / (299)) + (0.23877839753000815 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.06094153965376088 = (122.22104783186623 - 114.77270899889635) / 122.22104783186623 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorGivenSize<Int32>.List(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 12.758412649769076 (T) = (0 -127.23802198929593) / Math.Sqrt((16.42749918945691 / (299)) + (1.3035602885495106 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.06555316010467274 = (136.1640026558906 - 127.23802198929593) / 136.1640026558906 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorGivenSize<Int32>.Queue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 20.77398003313406 (T) = (0 -127.31380294965572) / Math.Sqrt((19.074394551299243 / (299)) + (0.30182788328467025 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.062055002537129526 = (135.73696036978498 - 127.31380294965572) / 135.73696036978498 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_GC<Char>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
NewOperator_Array - Duration of single invocation	120.96 ns	110.55 ns	0.91	0.02	False
AllocateArray - Duration of single invocation	258.67 ns	240.26 ns	0.93	0.09	False
AllocateUninitializedArray - Duration of single invocation	408.92 ns	342.21 ns	0.84	0.05	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_GC<Char>*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_GC<Char>.NewOperator_Array(length: 1000)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 19.407083577017143 (T) = (0 -111.13167238503213) / Math.Sqrt((8.200004084713607 / (299)) + (0.34299512110018177 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.06169339156099485 = (118.43854810946509 - 111.13167238503213) / 118.43854810946509 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_GC<Char>.AllocateArray(length: 1000, pinned: True)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 28.305878679721143 (T) = (0 -240.23662151080796) / Math.Sqrt((116.98169042518666 / (299)) + (0.4551130596087324 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.07988385003131188 = (261.0937994284562 - 240.23662151080796) / 261.0937994284562 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_GC<Char>.AllocateUninitializedArray(length: 10000, pinned: False)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 27.324316330322304 (T) = (0 -340.51353723136816) / Math.Sqrt((386.3110764612935 / (299)) + (6.4743412142783185 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.12971592911701676 = (391.26711452489974 - 340.51353723136816) / 391.26711452489974 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.AddGivenSize<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
HashSet - Duration of single invocation	15.73 μs	14.78 μs	0.94	0.00	False
Dictionary - Duration of single invocation	19.22 μs	16.86 μs	0.88	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<String>.HashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 10.061050081863256 (T) = (0 -14652.572533084904) / Math.Sqrt((20015.910252750793 / (299)) + (23545.943580396983 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.0575729944508777 = (15547.700189838379 - 14652.572533084904) / 15547.700189838379 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.AddGivenSize<String>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 132.28871309865184 (T) = (0 -16843.771555624615) / Math.Sqrt((44867.65194562229 / (299)) + (495.8903572780083 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.12239977062598441 = (19192.989007807264 - 16843.771555624615) / 19192.989007807264 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IndexerSet<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedList - Duration of single invocation	30.67 μs	28.39 μs	0.93	0.00	False
Dictionary - Duration of single invocation	6.92 μs	4.69 μs	0.68	0.00	False
Span - Duration of single invocation	189.96 ns	170.54 ns	0.90	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSet<Int32>.SortedList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/13/2023 5:42:59 PM, 2/20/2023 3:29:46 AM, 3/13/2023 10:19:02 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 105.94947530886995 (T) = (0 -28362.25368228293) / Math.Sqrt((71509.39111958521 / (299)) + (731.3157472544505 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.07586432397424722 = (30690.57327626919 - 28362.25368228293) / 30690.57327626919 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.IndexerSet<Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 179.16311555416104 (T) = (0 -4701.067038150609) / Math.Sqrt((17543.558577613672 / (299)) + (148.0183073562477 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.28371258080433015 = (6563.10150390399 - 4701.067038150609) / 6563.10150390399 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline

System.Collections.IndexerSet<Int32>.Span(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 139.33251824924056 (T) = (0 -170.67382769512264) / Math.Sqrt((0.7408278964482158 / (299)) + (0.03940769356855256 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09256607701127706 = (188.08402834775185 - 170.67382769512264) / 188.08402834775185 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Single

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToStringWithFormat - Duration of single invocation	203.10 ns	183.18 ns	0.90	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "G17")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 67.08506139832215 (T) = (0 -182.9994164098205) / Math.Sqrt((8.361160333597972 / (299)) + (0.03327602050506483 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.06755273084513849 = (196.25712087256682 - 182.9994164098205) / 196.25712087256682 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
CtorInvoke - Duration of single invocation	316.42 μs	268.77 μs	0.85	0.01	False
CtorInvoke - Duration of single invocation	304.14 μs	253.40 μs	0.83	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Common.CtorInvoke(Options: IgnoreCase, Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/21/2023 1:51:00 AM, 3/14/2023 10:43:28 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 42.24360652094735 (T) = (0 -267560.8123911053) / Math.Sqrt((17329781.61458531 / (299)) + (3451289.3578648265 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.14788962378108791 = (313997.8339171959 - 267560.8123911053) / 313997.8339171959 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Common.CtorInvoke(Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/21/2023 1:51:00 AM, 3/22/2023 1:13:01 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 95.61696032890602 (T) = (0 -253310.6182795699) / Math.Sqrt((18793025.29921117 / (299)) + (442605.41319806257 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.14758527576582636 = (297168.2809763161 - 253310.6182795699) / 297168.2809763161 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in BenchmarksGame.RegexRedux_1

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
RunBench - Duration of single invocation	48.45 ms	44.65 ms	0.92	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BenchmarksGame.RegexRedux_1*'

Payloads

Baseline
Compare

Histogram

BenchmarksGame.RegexRedux_1.RunBench

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 9:23:19 AM, 4/5/2023 12:23:38 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 23.03434085606672 (T) = (0 -44884609.761904754) / Math.Sqrt((273567496661.54248 / (299)) + (57252283482.31343 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.06766365893938314 = (48142078.974251345 - 44884609.761904754) / 48142078.974251345 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CtorGivenSize<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Queue - Duration of single invocation	268.91 ns	253.40 ns	0.94	0.03	False
HashSet - Duration of single invocation	767.36 ns	708.12 ns	0.92	0.06	False
List - Duration of single invocation	267.50 ns	250.77 ns	0.94	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorGivenSize<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorGivenSize<String>.Queue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 16.560968776538335 (T) = (0 -251.74054212249123) / Math.Sqrt((49.357806386204075 / (299)) + (2.2465403919001186 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.059169613400500615 = (267.5727163026403 - 251.74054212249123) / 267.5727163026403 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorGivenSize<String>.HashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/26/2023 10:10:21 PM, 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 33.619450081179316 (T) = (0 -708.6180654242886) / Math.Sqrt((382.50396444609277 / (299)) + (0.3882297382950949 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.05330823429882751 = (748.520364386445 - 708.6180654242886) / 748.520364386445 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorGivenSize<String>.List(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 17.30075030552457 (T) = (0 -251.42805186736106) / Math.Sqrt((47.528755784210276 / (299)) + (2.093942912098971 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.0598836192644137 = (267.4435389272052 - 251.42805186736106) / 267.4435389272052 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.AddGivenSize<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	7.83 μs	6.27 μs	0.80	0.03	False
HashSet - Duration of single invocation	7.31 μs	5.72 μs	0.78	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/5/2023 2:25:12 AM, 1/13/2023 8:11:05 PM, 3/5/2023 5:50:49 PM, 4/19/2023 6:20:25 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 29.190433322128026 (T) = (0 -6184.642055881661) / Math.Sqrt((38629.281278521805 / (299)) + (6639.810892781488 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.18595608089065532 = (7597.430446564053 - 6184.642055881661) / 7597.430446564053 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

System.Collections.AddGivenSize<Int32>.HashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 70.60462051045046 (T) = (0 -5765.146628697963) / Math.Sqrt((4495.535966357474 / (299)) + (1455.9785651038649 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.21503726044183374 = (7344.484442590236 - 5765.146628697963) / 7344.484442590236 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CreateAddAndRemove<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
HashSet - Duration of single invocation	19.64 μs	16.40 μs	0.83	0.00	False
Dictionary - Duration of single invocation	19.55 μs	17.60 μs	0.90	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndRemove<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndRemove<Int32>.HashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 75.02412185966872 (T) = (0 -16317.545829078628) / Math.Sqrt((27960.876817207583 / (299)) + (5424.60017634527 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.16701514868152645 = (19589.246795125655 - 16317.545829078628) / 19589.246795125655 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CreateAddAndRemove<Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 32.83220988144043 (T) = (0 -17625.20135887216) / Math.Sqrt((29210.20531062621 / (299)) + (12437.993421813168 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10821735414268752 = (19764.010256028505 - 17625.20135887216) / 19764.010256028505 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsKeyFalse<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	7.00 μs	5.32 μs	0.76	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse<Int32, Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyFalse<Int32, Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/31/2023 1:54:46 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 76.11863216059639 (T) = (0 -5297.968560430047) / Math.Sqrt((85317.59397053227 / (299)) + (975.2084779939735 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.261977036211803 = (7178.595816634365 - 5297.968560430047) / 7178.595816634365 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SingleSerial - Duration of single invocation	2.11 μs	1.90 μs	0.90	0.03	False
SingleParallel - Duration of single invocation	3.01 μs	2.73 μs	0.91	0.11	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests<Byte>*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleSerial(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: False)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/11/2023 5:39:19 AM, 3/17/2023 1:30:52 PM, 4/8/2023 3:10:58 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 13.504815157336006 (T) = (0 -1876.451666884532) / Math.Sqrt((8148.314030760085 / (299)) + (507.37842973312974 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09161520680003929 = (2065.701320554221 - 1876.451666884532) / 2065.701320554221 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline

System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleParallel(RentalSize: 4096, ManipulateArray: True, Async: False, UseSharedPool: True)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/16/2023 5:39:51 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 10.663832395383407 (T) = (0 -2731.468296491228) / Math.Sqrt((9694.596560509477 / (299)) + (1162.5173148196288 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.0740764717719945 = (2949.993399259008 - 2731.468296491228) / 2949.993399259008 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.TryAddGiventSize<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	7.72 μs	6.19 μs	0.80	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddGiventSize<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryAddGiventSize<Int32>.Dictionary(Count: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/31/2023 1:54:46 PM, 3/6/2023 6:03:21 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 66.61881205070596 (T) = (0 -6180.124590439049) / Math.Sqrt((38169.000543180955 / (299)) + (820.1325135650135 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.17754278809955326 = (7514.220194092132 - 6180.124590439049) / 7514.220194092132 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SelectToList - Duration of single invocation	472.65 ns	274.45 ns	0.58	0.25	False
SelectToList - Duration of single invocation	447.22 ns	296.68 ns	0.66	0.28	False
SelectToList - Duration of single invocation	528.75 ns	434.74 ns	0.82	0.36	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.SelectToList(input: Range)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/17/2023 2:42:45 PM, 4/18/2023 7:42:00 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 2.1131470688822906 (T) = (0 -341.17900151317866) / Math.Sqrt((672.5779629917915 / (299)) + (3340.5734278961186 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.17142277487916616 = (411.7648798075804 - 341.17900151317866) / 411.7648798075804 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToList(input: List)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/26/2023 2:33:00 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 37.262285594953816 (T) = (0 -288.6742405032266) / Math.Sqrt((1287.3622325129827 / (299)) + (48.08527421961074 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.3679153679907198 = (456.70188117939927 - 288.6742405032266) / 456.70188117939927 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.SelectToList(input: IList)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 39.77381994320641 (T) = (0 -437.8409659651873) / Math.Sqrt((1865.420286108145 / (299)) + (7.302115777339906 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.21106045327747835 = (554.9740379780716 - 437.8409659651873) / 554.9740379780716 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Memory.Span<Char>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SequenceEqual - Duration of single invocation	57.89 ns	51.84 ns	0.90	0.00	False
EndsWith - Duration of single invocation	30.71 ns	27.58 ns	0.90	0.01	False
StartsWith - Duration of single invocation	29.26 ns	26.40 ns	0.90	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Char>*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Char>.SequenceEqual(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 35.0583692145605 (T) = (0 -51.40711405284614) / Math.Sqrt((0.27546431041081715 / (299)) + (0.1392734614075883 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.12921683885293017 = (59.035493962846395 - 51.40711405284614) / 59.035493962846395 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

System.Memory.Span<Char>.EndsWith(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 8:11:05 PM, 2/9/2023 1:41:04 PM, 3/3/2023 4:19:26 AM, 3/16/2023 2:30:29 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 17.190137677011702 (T) = (0 -27.401878008237485) / Math.Sqrt((0.3597453207669241 / (299)) + (0.07354623731967011 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.0914093713976397 = (30.158662378444767 - 27.401878008237485) / 30.158662378444767 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.StartsWith(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/11/2023 4:58:32 PM, 1/21/2023 9:40:41 PM, 2/1/2023 6:25:50 PM, 2/9/2023 1:41:04 PM, 3/4/2023 8:55:37 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 24.212900610388367 (T) = (0 -26.493865173808647) / Math.Sqrt((0.2785834268590239 / (299)) + (0.03308631045069767 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09086646107915984 = (29.141885146221092 - 26.493865173808647) / 29.141885146221092 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IDictionary - Duration of single invocation	8.66 μs	7.70 μs	0.89	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse<Int32, Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<Int32, Int32>.IDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 3/6/2023 6:03:21 AM, 3/9/2023 9:18:02 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 117.79716552437056 (T) = (0 -7694.7200700172425) / Math.Sqrt((11464.512192505807 / (299)) + (48.29203030532513 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10149031669024129 = (8563.869942584146 - 7694.7200700172425) / 8563.869942584146 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.TryAddDefaultSize<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	27.48 μs	24.04 μs	0.87	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddDefaultSize<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryAddDefaultSize<String>.Dictionary(Count: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 34.83616461743375 (T) = (0 -24115.27417339895) / Math.Sqrt((90490.24285590724 / (299)) + (22584.607373058097 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.11334378656239166 = (27197.998286058228 - 24115.27417339895) / 27197.998286058228 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Net.NetworkInformation.Tests.PhysicalAddressTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
PAMedium - Duration of single invocation	38.56 ns	26.52 ns	0.69	0.02	False
PALong - Duration of single invocation	448.95 ns	185.77 ns	0.41	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.NetworkInformation.Tests.PhysicalAddressTests*'

Payloads

Baseline
Compare

Histogram

System.Net.NetworkInformation.Tests.PhysicalAddressTests.PAMedium

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/18/2023 9:46:56 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 22.069688939325907 (T) = (0 -26.287451450031067) / Math.Sqrt((17.96029729846852 / (299)) + (0.32602389773210755 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.2564357147687784 = (35.35330027565353 - 26.287451450031067) / 35.35330027565353 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Net.NetworkInformation.Tests.PhysicalAddressTests.PALong

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/18/2023 9:46:56 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 141.95789128801317 (T) = (0 -188.19360143220567) / Math.Sqrt((249.99900220107295 / (299)) + (6.3763500711125225 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.5648620710692354 = (432.4918351627982 - 188.19360143220567) / 432.4918351627982 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Tests.StreamReaderReadLineTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ReadLineAsync - Duration of single invocation	402.12 μs	350.87 μs	0.87	0.18	False
ReadLineAsync - Duration of single invocation	218.51 μs	196.66 μs	0.90	0.16	False
ReadLineAsync - Duration of single invocation	66.70 μs	61.42 μs	0.92	0.11	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 1, 1])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 23.05776458313149 (T) = (0 -348185.8104391917) / Math.Sqrt((303132985.27832973 / (299)) + (5801386.935845206 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1020881767220215 = (387772.83182226145 - 348185.8104391917) / 387772.83182226145 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 1, 8])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 25.300133720625865 (T) = (0 -195314.25240384616) / Math.Sqrt((98443976.51636964 / (299)) + (1497476.8670109338 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10546471748666505 = (218341.58609717508 - 195314.25240384616) / 218341.58609717508 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.StreamReaderReadLineTests.ReadLineAsync(LineLengthRange: [ 9, 32])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 39.38484468538464 (T) = (0 -61303.64949097433) / Math.Sqrt((2917014.2921220204 / (299)) + (10304.325918551354 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.06871608899233075 = (65827.02521365634 - 61303.64949097433) / 65827.02521365634 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Byte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
FirstArray - Duration of single invocation	6.44 ns	5.34 ns	0.83	0.01	False
IterateTryGetTenSegments - Duration of single invocation	53.87 ns	43.73 ns	0.81	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests<Byte>*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.FirstArray

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/27/2023 3:43:27 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 47.10911025718908 (T) = (0 -5.286666666669032) / Math.Sqrt((0.008338891324769403 / (299)) + (0.002425711357139493 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.20490948528117237 = (6.649138140628663 - 5.286666666669032) / 6.649138140628663 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateTryGetTenSegments

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/23/2023 10:43:32 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 15.083094428400017 (T) = (0 -44.78072427126789) / Math.Sqrt((1.1079201331017086 / (299)) + (0.8329357705821762 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.15157772357866725 = (52.78117455868103 - 44.78072427126789) / 52.78117455868103 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CreateAddAndClear<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	14.34 μs	11.27 μs	0.79	0.01	False
HashSet - Duration of single invocation	13.17 μs	10.77 μs	0.82	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 8.458708884175119 (T) = (0 -11762.194457711781) / Math.Sqrt((48013.19206767874 / (299)) + (181080.13952940181 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1503217813492045 = (13843.116369852314 - 11762.194457711781) / 13843.116369852314 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline

System.Collections.CreateAddAndClear<Int32>.HashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 123.01193080753819 (T) = (0 -10800.536536765272) / Math.Sqrt((13099.611499791115 / (299)) + (966.8773380473251 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.17893036488346656 = (13154.227211474414 - 10800.536536765272) / 13154.227211474414 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Tests.Perf_Encoding

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetString - Duration of single invocation	27.25 ns	23.33 ns	0.86	0.16	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetString(size: 16, encName: "ascii")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/17/2023 4:06:07 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 20.84169691761861 (T) = (0 -23.630660060206775) / Math.Sqrt((0.609949847713902 / (299)) + (0.08212039037567463 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1313880469891774 = (27.205082751045616 - 23.630660060206775) / 27.205082751045616 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Char>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
FirstArray - Duration of single invocation	6.58 ns	5.57 ns	0.85	0.00	False
IterateTryGetTenSegments - Duration of single invocation	54.59 ns	46.30 ns	0.85	0.05	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests<Char>*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Char>.FirstArray

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/27/2023 2:26:56 PM, 4/8/2023 3:10:58 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 34.90433736389261 (T) = (0 -5.635100308719889) / Math.Sqrt((0.015804708304201612 / (299)) + (0.003259890104137687 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.172931535874985 = (6.813342006313179 - 5.635100308719889) / 6.813342006313179 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateTryGetTenSegments

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/24/2023 7:13:05 PM, 3/25/2023 2:21:39 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 19.41885933889532 (T) = (0 -46.84390880994468) / Math.Sqrt((3.241836553231757 / (299)) + (0.3838227619432126 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.13377360334993543 = (54.07813591354748 - 46.84390880994468) / 54.07813591354748 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_GC<Byte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
AllocateUninitializedArray - Duration of single invocation	64.73 ns	59.00 ns	0.91	0.03	False
NewOperator_Array - Duration of single invocation	68.49 ns	58.19 ns	0.85	0.05	False
AllocateUninitializedArray - Duration of single invocation	252.83 ns	205.52 ns	0.81	0.06	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_GC<Byte>*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_GC<Byte>.AllocateUninitializedArray(length: 1000, pinned: False)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 8.395152654803406 (T) = (0 -59.16876044521701) / Math.Sqrt((3.084143203217368 / (299)) + (0.8764619358766688 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.07238432437877743 = (63.78585657857905 - 59.16876044521701) / 63.78585657857905 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_GC<Byte>.NewOperator_Array(length: 1000)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 3:00:37 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 11.700592686702164 (T) = (0 -57.75591998137917) / Math.Sqrt((3.321863154533336 / (299)) + (0.552715478567296 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.0821813317583744 = (62.92737550438918 - 57.75591998137917) / 62.92737550438918 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_GC<Byte>.AllocateUninitializedArray(length: 10000, pinned: False)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 27.341301481989294 (T) = (0 -204.30199872538313) / Math.Sqrt((252.98920584209168 / (299)) + (2.8221224789054955 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1517448949618743 = (240.84971314873556 - 204.30199872538313) / 240.84971314873556 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.TryAddDefaultSize<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	13.40 μs	11.06 μs	0.83	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddDefaultSize<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryAddDefaultSize<Int32>.Dictionary(Count: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/19/2023 6:20:25 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 38.087591003423334 (T) = (0 -11168.134458695165) / Math.Sqrt((52029.52199581927 / (299)) + (11466.255265305957 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.17735160706040778 = (13575.829667384096 - 11168.134458695165) / 13575.829667384096 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CtorFromCollection<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Stack - Duration of single invocation	211.85 ns	197.97 ns	0.93	0.02	False
ImmutableArray - Duration of single invocation	219.80 ns	203.71 ns	0.93	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<Int32>.Stack(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 12:47:31 AM, 3/10/2023 8:06:53 PM, 4/6/2023 11:23:56 PM, 4/12/2023 2:59:13 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 15.48245656949194 (T) = (0 -198.36189446000654) / Math.Sqrt((24.24159070981019 / (299)) + (2.2866968545807835 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.06688228845960664 = (212.57971208428802 - 198.36189446000654) / 212.57971208428802 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorFromCollection<Int32>.ImmutableArray(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/11/2023 4:58:32 PM, 1/21/2023 9:40:41 PM, 2/1/2023 6:25:50 PM, 2/7/2023 3:56:56 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 47.678778753033455 (T) = (0 -203.79904686110808) / Math.Sqrt((19.302918152146518 / (299)) + (0.044450532842966434 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.061836332545394605 = (217.23186894888917 - 203.79904686110808) / 217.23186894888917 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Memory.Span<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
EndsWith - Duration of single invocation	60.13 ns	50.83 ns	0.85	0.00	False
StartsWith - Duration of single invocation	58.52 ns	51.62 ns	0.88	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.EndsWith(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 35.268606109944685 (T) = (0 -50.94984033598353) / Math.Sqrt((0.45265027721430884 / (299)) + (0.20544703186904595 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.15479135637396343 = (60.28078477451816 - 50.94984033598353) / 60.28078477451816 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

System.Memory.Span<Int32>.StartsWith(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 226.84943502275775 (T) = (0 -51.608202209441636) / Math.Sqrt((0.3256707013567913 / (299)) + (9.156874091500086E-05 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1282265168249809 = (59.19909610175728 - 51.608202209441636) / 59.19909610175728 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsTrue<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
HashSet - Duration of single invocation	6.36 μs	4.39 μs	0.69	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<Int32>.HashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/16/2023 10:20:48 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 34.01900411865289 (T) = (0 -4439.1489112351) / Math.Sqrt((25675.403943638128 / (299)) + (10441.90916588956 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.31396920440199294 = (6470.772069882858 - 4439.1489112351) / 6470.772069882858 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Decimal

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Divide - Duration of single invocation	48.15 ns	44.29 ns	0.92	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Decimal*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Decimal.Divide

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 3:30:06 PM, 4/8/2023 3:10:58 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 41.313083650825185 (T) = (0 -44.37815601600132) / Math.Sqrt((0.4649338655116835 / (299)) + (0.006449123427680219 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.053623727630033764 = (46.892718373916075 - 44.37815601600132) / 46.892718373916075 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Memory.Span<Byte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SequenceEqual - Duration of single invocation	29.01 ns	26.55 ns	0.92	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Byte>*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.SequenceEqual(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/1/2023 6:25:50 PM, 2/9/2023 1:41:04 PM, 3/3/2023 4:19:26 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 30.444700782759938 (T) = (0 -26.39154823090367) / Math.Sqrt((0.2720582304942836 / (299)) + (0.01866453496040131 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08876929111310664 = (28.96253163278711 - 26.39154823090367) / 28.96253163278711 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CreateAddAndClear<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
HashSet - Duration of single invocation	22.06 μs	20.45 μs	0.93	0.00	False
Dictionary - Duration of single invocation	27.80 μs	24.88 μs	0.89	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<String>.HashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/7/2023 1:52:33 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 20.208036872214876 (T) = (0 -20540.755481050186) / Math.Sqrt((54416.85365621003 / (299)) + (14187.841440399801 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.06449847019800634 = (21956.944833000754 - 20540.755481050186) / 21956.944833000754 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CreateAddAndClear<String>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/14/2023 3:18:57 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 21.85709636325238 (T) = (0 -24644.437722841558) / Math.Sqrt((100872.35820403017 / (299)) + (55185.273086922694 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1082469183614519 = (27635.94343577567 - 24644.437722841558) / 27635.94343577567 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CtorDefaultSize<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentBag - Duration of single invocation	716.37 ns	646.63 ns	0.90	0.17	False
LinkedList - Duration of single invocation	6.95 ns	5.02 ns	0.72	0.64	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorDefaultSize<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorDefaultSize<Int32>.ConcurrentBag

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/26/2023 7:41:12 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 7.326046422532934 (T) = (0 -651.7354595992166) / Math.Sqrt((413.33537956777565 / (299)) + (354.1926996787008 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10941135850553912 = (731.803022443185 - 651.7354595992166) / 731.803022443185 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorDefaultSize<Int32>.LinkedList

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/16/2023 3:50:13 PM, 4/20/2023 12:14:19 AM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 7.343053461866365 (T) = (0 -5.247256097977044) / Math.Sqrt((0.3939626755006197 / (299)) + (0.2041555825094146 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.12180901391795264 = (5.97507396584324 - 5.247256097977044) / 5.97507396584324 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Globalization.Tests.StringSearch

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
LastIndexOf_Word_NotFound - Duration of single invocation	595.94 ns	533.36 ns	0.89	0.00	False
LastIndexOf_Word_NotFound - Duration of single invocation	595.29 ns	533.09 ns	0.90	0.00	False
IndexOf_Word_NotFound - Duration of single invocation	603.48 ns	531.18 ns	0.88	0.00	False
IndexOf_Word_NotFound - Duration of single invocation	606.60 ns	532.27 ns	0.88	0.00	False
IndexOf_Word_NotFound - Duration of single invocation	602.87 ns	533.12 ns	0.88	0.00	False
LastIndexOf_Word_NotFound - Duration of single invocation	596.10 ns	532.65 ns	0.89	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, None, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 314.377326189414 (T) = (0 -532.9866761941581) / Math.Sqrt((0.8576752947963743 / (299)) + (0.11369191607313274 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10641845662920148 = (596.4611513613047 - 532.9866761941581) / 596.4611513613047 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, None, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 189.9656316847461 (T) = (0 -532.5083135416671) / Math.Sqrt((1.3188149139482808 / (299)) + (0.32754145347711283 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10732798539048025 = (596.5329984883658 - 532.5083135416671) / 596.5329984883658 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, None, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 216.54750038025716 (T) = (0 -531.699528498482) / Math.Sqrt((1.5031281407513328 / (299)) + (0.3139948606292048 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.11885610330534392 = (603.4196349688075 - 531.699528498482) / 603.4196349688075 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, None, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 241.29259334810527 (T) = (0 -532.6328283359046) / Math.Sqrt((1.400664539825276 / (299)) + (0.243447405917324 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.11717130021017436 = (603.3252299825641 - 532.6328283359046) / 603.3252299825641 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, IgnoreNonSpace, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 147.77329358285286 (T) = (0 -532.1824599671868) / Math.Sqrt((1.4881752518733262 / (299)) + (0.6803319598914297 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.11791281122492166 = (603.3218334189943 - 532.1824599671868) / 603.3218334189943 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreNonSpace, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 313.93184543286196 (T) = (0 -533.0300649539448) / Math.Sqrt((0.9716599302969746 / (299)) + (0.1117969316728972 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10598365320427132 = (596.2195958322174 - 533.0300649539448) / 596.2195958322174 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Memory.ReadOnlySpan

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Trim - Duration of single invocation	3.81 ns	2.41 ns	0.63	0.13	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.Trim(input: "abcdefg")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 65.96897231191404 (T) = (0 -2.3733938247660653) / Math.Sqrt((0.07701870020622006 / (299)) + (0.0018407133763492108 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4506661411805388 = (4.320494334477355 - 2.3733938247660653) / 4.320494334477355 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Char

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Char_ToLower - Duration of single invocation	20.16 ns	18.23 ns	0.90	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Char*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Char.Char_ToLower(c: '你', cultureName: zh-Hans)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/9/2023 9:43:16 AM, 3/10/2023 12:53:36 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 12.48210520949413 (T) = (0 -18.138359998850028) / Math.Sqrt((0.25419346044751595 / (299)) + (0.059951146999309986 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.09035394954545663 = (19.940019516147434 - 18.138359998850028) / 19.940019516147434 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsKeyTrue<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedList - Duration of single invocation	30.36 μs	28.42 μs	0.94	0.00	False
Dictionary - Duration of single invocation	5.88 μs	4.47 μs	0.76	0.06	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue<Int32, Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyTrue<Int32, Int32>.SortedList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/5/2023 2:25:12 AM, 1/27/2023 3:43:27 AM, 3/3/2023 4:59:02 PM, 3/8/2023 8:26:35 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 99.14068619455117 (T) = (0 -28398.703897451178) / Math.Sqrt((55436.86879580944 / (299)) + (697.66290571023 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.06661650731625524 = (30425.54761258609 - 28398.703897451178) / 30425.54761258609 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsKeyTrue<Int32, Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/20/2023 1:51:38 AM, 3/25/2023 12:18:49 AM, 4/9/2023 11:04:55 PM, 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 19.138844443152372 (T) = (0 -4535.840031112862) / Math.Sqrt((181141.51530267458 / (299)) + (3681.8963753098087 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1530136658572582 = (5355.269439740973 - 4535.840031112862) / 5355.269439740973 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_String

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TrimEnd - Duration of single invocation	16.10 ns	12.86 ns	0.80	0.28	False
TrimStart - Duration of single invocation	14.37 ns	12.81 ns	0.89	0.33	False	151.66705269357004	131.5687434628964	0.867484012686127)

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.TrimEnd(s: "Test ")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/22/2023 1:13:01 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 56.49872571532466 (T) = (0 -12.809833353955996) / Math.Sqrt((0.38416599529700596 / (299)) + (0.004833557748432254 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1918236819972689 = (15.850295373184528 - 12.809833353955996) / 15.850295373184528 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.TrimStart(s: " Test")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/7/2023 10:47:28 PM, 3/16/2023 10:20:48 AM, 4/25/2023 1:02:54 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 23.751526338226803 (T) = (0 -12.838588389289638) / Math.Sqrt((0.41884249277055813 / (299)) + (0.027923640009873883 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.16068254363780413 = (15.296462967582224 - 12.838588389289638) / 15.296462967582224 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CtorGivenSizeNonGeneric

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedList - Duration of single invocation	584.48 ns	543.99 ns	0.93	0.04	False
Queue - Duration of single invocation	297.09 ns	252.69 ns	0.85	0.05	False
ArrayList - Duration of single invocation	266.60 ns	249.69 ns	0.94	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorGivenSizeNonGeneric*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorGivenSizeNonGeneric.SortedList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 3:00:37 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 21.367197762115296 (T) = (0 -544.8411727084144) / Math.Sqrt((236.4892056627848 / (299)) + (3.759234980329832 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.053091778255747894 = (575.3896314310059 - 544.8411727084144) / 575.3896314310059 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

System.Collections.CtorGivenSizeNonGeneric.Queue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 16.687729988808666 (T) = (0 -251.11978400620376) / Math.Sqrt((57.43862601637091 / (299)) + (2.2814891813850506 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.06090867918147546 = (267.40720357986527 - 251.11978400620376) / 267.40720357986527 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorGivenSizeNonGeneric.ArrayList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 26.968247976812595 (T) = (0 -249.59907339475683) / Math.Sqrt((42.11261083032516 / (299)) + (0.6613871381966653 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.06098468260781622 = (265.8093736829968 - 249.59907339475683) / 265.8093736829968 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Tests.Perf_StringBuilder

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToString_SingleSegment - Duration of single invocation	26.21 ns	23.79 ns	0.91	0.13	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_StringBuilder.ToString_SingleSegment(length: 100)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/25/2023 3:45:56 AM, 3/20/2023 7:45:43 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 10.890668267362962 (T) = (0 -23.920368078293986) / Math.Sqrt((0.48461867854994634 / (299)) + (0.06848713338582106 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.0664596143491528 = (25.62328148408614 - 23.920368078293986) / 25.62328148408614 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsFalse<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
HashSet - Duration of single invocation	7.53 μs	4.26 μs	0.57	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<Int32>.HashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/20/2023 3:29:46 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 184.6288291200355 (T) = (0 -4239.121858629114) / Math.Sqrt((35623.352610143906 / (299)) + (660.824916002724 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.4451837701815223 = (7640.587334685669 - 4239.121858629114) / 7640.587334685669 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Threading.Tests.Perf_CancellationToken

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
CreateTokenDispose - Duration of single invocation	14.39 ns	10.85 ns	0.75	0.21	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_CancellationToken*'

Payloads

Baseline
Compare

Histogram

System.Threading.Tests.Perf_CancellationToken.CreateTokenDispose

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 73.56912053744209 (T) = (0 -10.778670028799437) / Math.Sqrt((0.18362879470603463 / (299)) + (0.005472600415052521 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.25207764117361187 = (14.411482557779024 - 10.778670028799437) / 14.411482557779024 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsTrue<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Array - Duration of single invocation	462.13 μs	423.78 μs	0.92	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<String>.Array(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 107.43913471230964 (T) = (0 -423077.82238969736) / Math.Sqrt((839589.4575127463 / (299)) + (384660.60513373755 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.08418458354832908 = (461968.4434106967 - 423077.82238969736) / 461968.4434106967 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.TryAddGiventSize<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	19.10 μs	16.73 μs	0.88	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddGiventSize<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryAddGiventSize<String>.Dictionary(Count: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 90.09709862006869 (T) = (0 -16763.152074533762) / Math.Sqrt((48354.665799876384 / (299)) + (1685.0767685430487 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.1263018864581596 = (19186.43501080536 - 16763.152074533762) / 19186.43501080536 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsFalse<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Array - Duration of single invocation	906.33 μs	831.24 μs	0.92	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<String>.Array(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 253.53779681033282 (T) = (0 -830836.1528822057) / Math.Sqrt((6031813.366767501 / (299)) + (205982.23736901846 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.0833702261267755 = (906403.1919577548 - 830836.1528822057) / 906403.1919577548 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Tests.NonStandardArrayPoolTests<Object>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
RentNoReturn - Duration of single invocation	49.98 ns	46.99 ns	0.94	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.NonStandardArrayPoolTests<Object>*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.NonStandardArrayPoolTests<Object>.RentNoReturn(RentalSize: 64, UseSharedPool: True)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 3/15/2023 2:24:48 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 23.107511210252188 (T) = (0 -47.05065395299146) / Math.Sqrt((0.8441224259968939 / (299)) + (0.03544280081710819 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.05608572190671242 = (49.846320841797265 - 47.05065395299146) / 49.846320841797265 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IterateForEach<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentQueue - Duration of single invocation	7.80 μs	7.13 μs	0.91	0.11	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<Int32>.ConcurrentQueue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 24.050895536937578 (T) = (0 -7135.78049981816) / Math.Sqrt((34452.016536055366 / (299)) + (1053.7735312245625 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.06785628523600648 = (7655.236404855066 - 7135.78049981816) / 7655.236404855066 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Uri

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
PathAndQuery - Duration of single invocation	6.61 ns	3.74 ns	0.57	0.21	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Uri*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Uri.PathAndQuery

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/9/2023 10:41:51 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 10.702363820127966 (T) = (0 -3.080505355496113) / Math.Sqrt((0.030119475716091092 / (299)) + (0.35355810852364444 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.544045115021527 = (6.7561626314005885 - 3.080505355496113) / 6.7561626314005885 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Perf_Convert

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToHexString - Duration of single invocation	132.18 ns	59.95 ns	0.45	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Perf_Convert*'

Payloads

Baseline
Compare

Histogram

System.Perf_Convert.ToHexString

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/18/2023 9:46:56 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 139.8025001953871 (T) = (0 -60.312708089863804) / Math.Sqrt((43.089392797793785 / (299)) + (0.186847331493082 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.512923128799651 = (123.82585102271345 - 60.312708089863804) / 123.82585102271345 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
CertProp - Duration of single invocation	60.27 ns	41.99 ns	0.70	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests*'

Payloads

Baseline
Compare

Histogram

System.Security.Cryptography.X509Certificates.Tests.X509Certificate2Tests.CertProp

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/18/2023 9:46:56 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 26.838162496317555 (T) = (0 -40.93624141966498) / Math.Sqrt((17.679110045078136 / (299)) + (0.9234139470590516 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.28424932249403084 = (57.19343719283253 - 40.93624141966498) / 57.19343719283253 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Tests.BinaryWriterExtendedTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
WriteAsciiString - Duration of single invocation	702.96 ns	667.50 ns	0.95	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 8000)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 4:18:46 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 149.07976806210675 (T) = (0 -667.2018211736487) / Math.Sqrt((0.8627304432798436 / (299)) + (0.16762922940879676 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.051380807192129514 = (703.3399980014752 - 667.2018211736487) / 703.3399980014752 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Compare

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Array

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ArrayCreate1D - Duration of single invocation	1.08 μs	923.13 ns	0.85	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Array*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Array.ArrayCreate1D

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 2/27/2023 8:52:52 AM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 28.643801932735318 (T) = (0 -922.4034956829474) / Math.Sqrt((614.3346805822331 / (299)) + (2.6454807469673054 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.05052487044564256 = (971.4877904341568 - 922.4034956829474) / 971.4877904341568 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	195bf987887ea645a747ff0586440f7920c47b01
Compare	63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Perf_Utf8Encoding

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetString - Duration of single invocation	274.00 μs	252.56 μs	0.92	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Utf8Encoding.GetString(Input: Chinese)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/17/2023 4:06:07 PM, 3/16/2023 8:14:39 PM, 4/25/2023 6:46:56 PM, 4/27/2023 1:05:59 AM falls between 4/18/2023 7:36:21 AM and 4/27/2023 1:05:59 AM.
IsImprovementStdDev: Marked as improvement because 60.9979044237311 (T) = (0 -252949.54973118278) / Math.Sqrt((50217655.20478146 / (299)) + (231274.3140012735 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.10664195915721218 = (283144.6499239565 - 252949.54973118278) / 283144.6499239565 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[AndyAyersMS]

Suspect many of these are from dotnet/runtime#85265
But the EH/GC ones are more likely dotnet/runtime#85355