[Perf] Linux/x64: 7 Regressions on 1/15/2023 10:25:39 PM

[performanceautofiler]

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	191b98d3a63b7b2d8fd2c8cdf7c1f083160251d8
Compare	2908445dae75580421a3585a45ba3d570bd75cd3
Diff	Diff

Regressions in System.Collections.CopyTo<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Array - Duration of single invocation	10.74 μs	12.07 μs	1.12	0.46	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CopyTo<String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CopyTo<String>.Array(Size: 2048)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.072850654163446 > 10.4494441908901.
IsChangePoint: Marked as a change because one of 11/24/2022 8:21:24 PM, 12/14/2022 11:14:18 PM, 1/15/2023 6:48:05 PM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -12.681376212285942 (T) = (0 -11983.61751735723) / Math.Sqrt((741905.2110877576 / (48)) + (15620.702014191123 / (9))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (9) - 2, .025) and -0.16110673380451423 = (10320.85782337286 - 11983.61751735723) / 10320.85782337286 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	191b98d3a63b7b2d8fd2c8cdf7c1f083160251d8
Compare	2908445dae75580421a3585a45ba3d570bd75cd3
Diff	Diff

Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetValue - Duration of single invocation	16.13 ns	26.76 ns	1.66	0.70	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Primitives.StringSegmentBenchmark.GetValue

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.760579983633118 > 17.191122567728367.
IsChangePoint: Marked as a change because one of 1/15/2023 6:48:05 PM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -9.029944649446318 (T) = (0 -23.64007773746966) / Math.Sqrt((6.340326585144901 / (48)) + (3.6277265206921276 / (9))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (9) - 2, .025) and -0.38780185397169037 = (17.034188036148777 - 23.64007773746966) / 17.034188036148777 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	191b98d3a63b7b2d8fd2c8cdf7c1f083160251d8
Compare	2908445dae75580421a3585a45ba3d570bd75cd3
Diff	Diff

Regressions in System.Memory.Span<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SequenceCompareTo - Duration of single invocation	22.13 ns	26.31 ns	1.19	0.62	False
Reverse - Duration of single invocation	165.41 ns	181.96 ns	1.10	0.01	False
SequenceEqual - Duration of single invocation	293.54 ns	343.35 ns	1.17	0.00	False
StartsWith - Duration of single invocation	149.94 ns	176.59 ns	1.18	0.03	False
EndsWith - Duration of single invocation	151.26 ns	180.99 ns	1.20	0.01	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.SequenceCompareTo(Size: 4)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.312439779835426 > 24.191300824994723.
IsChangePoint: Marked as a change because one of 1/15/2023 6:48:05 PM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -5.870242255845162 (T) = (0 -27.522525662323424) / Math.Sqrt((5.308472408770693 / (48)) + (1.9486017086800658 / (9))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (9) - 2, .025) and -0.13893436468167636 = (24.16515517996137 - 27.522525662323424) / 24.16515517996137 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span<Int32>.Reverse(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 181.95694441033578 > 173.7995406284211.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -50.51740625270476 (T) = (0 -182.53042579164222) / Math.Sqrt((0.9138595753631271 / (32)) + (2.0009853139507703 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.10035742715164268 = (165.8828497792175 - 182.53042579164222) / 165.8828497792175 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span<Int32>.SequenceEqual(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 343.34756124679626 > 305.8587698276505.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -105.1614766004193 (T) = (0 -344.7834951205122) / Math.Sqrt((4.226453323000013 / (32)) + (3.2336142982785088 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.1847615478136906 = (291.0150956171402 - 344.7834951205122) / 291.0150956171402 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span<Int32>.StartsWith(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 176.59061864004147 > 159.84796660224285.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -53.198567627023515 (T) = (0 -176.57980354561616) / Math.Sqrt((6.0620377575141315 / (32)) + (0.4582313092911449 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.1591844814347065 = (152.33106237504646 - 176.57980354561616) / 152.33106237504646 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span<Int32>.EndsWith(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 180.9865922212758 > 158.42406336186522.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -66.81280140786993 (T) = (0 -177.5978575850546) / Math.Sqrt((1.899932918204738 / (32)) + (2.427253129269053 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.17482380285805468 = (151.16978150511005 - 177.5978575850546) / 151.16978150511005 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[radekdoulik]

This looks like effect of dotnet/runtime#80668

cc @EgorBo

[EgorBo]

This looks like effect of dotnet/runtime#80668

cc @EgorBo

dotnet/runtime#80762