Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/arm64: 87 Regressions on 1/14/2023 12:48:25 AM #11801

Closed
performanceautofiler bot opened this issue Jan 17, 2023 · 4 comments
Closed

[Perf] Linux/arm64: 87 Regressions on 1/14/2023 12:48:25 AM #11801

performanceautofiler bot opened this issue Jan 17, 2023 · 4 comments
Labels
ampere arch-arm64 branch-refs/heads/main kind-micro_mono mono-interpreter os-linux perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-mono

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6aaaaaa5c7b1160a237bb381074e707c3ea1e9b4
Compare d0fcd6206daac01f6279adc51e40d7e1df4fed19
Diff Diff

Regressions in System.Tests.Perf_Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringWithFormat - Duration of single invocation 2.65 μs 3.45 μs 1.30 0.01 False
ToStringWithFormat - Duration of single invocation 4.20 μs 4.99 μs 1.19 0.01 False
ToStringWithFormat - Duration of single invocation 4.51 μs 5.27 μs 1.17 0.03 False
ToStringWithCultureInfo - Duration of single invocation 4.37 μs 5.17 μs 1.18 0.03 False
ToStringWithFormat - Duration of single invocation 2.68 μs 3.48 μs 1.30 0.03 False
ToString - Duration of single invocation 4.43 μs 5.22 μs 1.18 0.03 False
ToStringWithCultureInfo - Duration of single invocation 4.44 μs 5.20 μs 1.17 0.03 False
ToStringWithFormat - Duration of single invocation 4.45 μs 5.25 μs 1.18 0.03 False
ToStringWithFormat - Duration of single invocation 2.73 μs 3.73 μs 1.37 0.01 False
ToStringWithFormat - Duration of single invocation 2.71 μs 3.53 μs 1.30 0.01 False
ToStringWithFormat - Duration of single invocation 3.62 μs 4.41 μs 1.22 0.01 False
ToStringWithFormat - Duration of single invocation 4.47 μs 5.26 μs 1.18 0.03 False
ToString - Duration of single invocation 2.60 μs 3.41 μs 1.31 0.01 False
ToStringWithFormat - Duration of single invocation 3.67 μs 4.46 μs 1.22 0.01 False
ToString - Duration of single invocation 4.39 μs 5.18 μs 1.18 0.03 False
ToStringWithFormat - Duration of single invocation 2.63 μs 3.43 μs 1.30 0.02 False
Parse - Duration of single invocation 2.73 μs 2.92 μs 1.07 0.03 False
TryParse - Duration of single invocation 2.74 μs 2.94 μs 1.07 0.01 False
ToStringWithFormat - Duration of single invocation 4.46 μs 5.23 μs 1.17 0.03 False
ToStringWithCultureInfo - Duration of single invocation 2.58 μs 3.40 μs 1.32 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "R")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -89.44956437413936 (T) = (0 -3442.7816615357106) / Math.Sqrt((325.79128979995613 / (8)) + (83.3847296505634 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.30868020113596145 = (2630.7280102100613 - 3442.7816615357106) / 2630.7280102100613 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "G17")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -16.767524921391644 (T) = (0 -4945.244735250852) / Math.Sqrt((3931.547234243576 / (8)) + (4314.295626404843 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.21138303302000566 = (4082.313026064302 - 4945.244735250852) / 4082.313026064302 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "R")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -27.419937422756853 (T) = (0 -5254.582496378453) / Math.Sqrt((4944.877457363194 / (8)) + (339.88542412356895 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.17163142233760526 = (4484.842584619881 - 5254.582496378453) / 4484.842584619881 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithCultureInfo(value: 1.7976931348623157E+308, culture: zh)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -40.48055686136657 (T) = (0 -5153.69236051082) / Math.Sqrt((1754.3727444995147 / (8)) + (330.98669169634786 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.18214169660191992 = (4359.6231951932405 - 5153.69236051082) / 4359.6231951932405 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -48.56190814715478 (T) = (0 -3478.015557492451) / Math.Sqrt((2146.344949192844 / (8)) + (31.14024484310453 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.3076068370456155 = (2659.832802152228 - 3478.015557492451) / 2659.832802152228 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToString(value: -1.7976931348623157E+308)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -28.523379092380544 (T) = (0 -5199.677704140542) / Math.Sqrt((2507.8338241203464 / (8)) + (947.1905292359944 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.18189034794816353 = (4399.458641123106 - 5199.677704140542) / 4399.458641123106 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithCultureInfo(value: -1.7976931348623157E+308, culture: zh)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -53.99764500330823 (T) = (0 -5203.267669531875) / Math.Sqrt((1756.8325006748998 / (8)) + (16.20124578680167 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.1856734110954737 = (4388.4493156715425 - 5203.267669531875) / 4388.4493156715425 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "R")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -38.95651464859978 (T) = (0 -5231.014007547706) / Math.Sqrt((1565.2264321780567 / (8)) + (487.92579712277 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.1850392158415606 = (4414.211730396516 - 5231.014007547706) / 4414.211730396516 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -74.54722383430469 (T) = (0 -3715.959675680817) / Math.Sqrt((508.1964524305023 / (8)) + (237.909355436855 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.371739665918528 = (2708.939435087765 - 3715.959675680817) / 2708.939435087765 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -34.84803813973433 (T) = (0 -3507.145792305866) / Math.Sqrt((846.4974756593191 / (8)) + (862.0122472668288 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.299068099706774 = (2699.739754288093 - 3507.145792305866) / 2699.739754288093 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G17")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -64.93182839540324 (T) = (0 -4420.780106970398) / Math.Sqrt((960.5726065747788 / (8)) + (76.63008469460875 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.22676712312880695 = (3603.6017134983404 - 4420.780106970398) / 3603.6017134983404 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -54.73426438736313 (T) = (0 -5253.450352887274) / Math.Sqrt((1377.5366514085 / (8)) + (100.01234214227154 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.18385902970511986 = (4437.564119602841 - 5253.450352887274) / 4437.564119602841 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToString(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -34.22751732969279 (T) = (0 -3387.910274160935) / Math.Sqrt((1324.1410786337244 / (8)) + (775.3452977300559 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.31167974800341325 = (2582.8791511936333 - 3387.910274160935) / 2582.8791511936333 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G17")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -67.66476179820079 (T) = (0 -4455.364272985304) / Math.Sqrt((955.7781571039461 / (8)) + (49.38902023459537 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.22302105877772635 = (3642.917054460162 - 4455.364272985304) / 3642.917054460162 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToString(value: 1.7976931348623157E+308)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 10/7/2022 12:48:42 AM, 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -38.92906669775108 (T) = (0 -5165.248816291789) / Math.Sqrt((1616.604039856695 / (8)) + (480.60953077953286 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.18838059872386312 = (4346.460066613732 - 5165.248816291789) / 4346.460066613732 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "G")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -49.845165451047215 (T) = (0 -3417.334024749168) / Math.Sqrt((544.6597291888352 / (8)) + (374.5173584086956 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.30390875705313436 = (2620.8383111655958 - 3417.334024749168) / 2620.8383111655958 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.Parse(value: "-1.7976931348623157e+308")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/3/2023 12:19:11 AM, 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -8.020060244752091 (T) = (0 -2936.8047170510827) / Math.Sqrt((1774.694749563291 / (8)) + (880.7929297851506 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.0755882182548717 = (2730.417335563615 - 2936.8047170510827) / 2730.417335563615 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.TryParse(value: "-1.7976931348623157e+308")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/3/2023 12:19:11 AM, 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -18.127839490195225 (T) = (0 -2944.5685352603828) / Math.Sqrt((1257.4689279717527 / (8)) + (14.254336482568505 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.08567567222141599 = (2712.1990577862543 - 2944.5685352603828) / 2712.1990577862543 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -19.980417396684775 (T) = (0 -5205.209080619873) / Math.Sqrt((5121.020461698551 / (8)) + (1760.4852070576644 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.17601701507236028 = (4426.134157846004 - 5205.209080619873) / 4426.134157846004 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithCultureInfo(value: 12345, culture: zh)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -41.5855783552208 (T) = (0 -3382.4793069384737) / Math.Sqrt((396.01951540222774 / (8)) + (677.3070002594644 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.3196449190843654 = (2563.1738189735192 - 3382.4793069384737) / 2563.1738189735192 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline 6aaaaaa5c7b1160a237bb381074e707c3ea1e9b4
Compare d0fcd6206daac01f6279adc51e40d7e1df4fed19
Diff Diff

Regressions in System.Perf_Convert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToDateTime_String - Duration of single invocation 6.67 μs 7.17 μs 1.08 0.01 False
ToDateTime_String - Duration of single invocation 4.48 μs 4.80 μs 1.07 0.01 False
ToDateTime_String - Duration of single invocation 3.68 μs 4.11 μs 1.12 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Perf_Convert*'

Payloads

Baseline
Compare

Histogram

System.Perf_Convert.ToDateTime_String(value: "Thursday, February 26, 2009")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/3/2023 12:19:11 AM, 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -10.075884076466101 (T) = (0 -7122.932397834369) / Math.Sqrt((3868.7307629645866 / (8)) + (4130.85161299462 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.07691139545212554 = (6614.2232572847 - 7122.932397834369) / 6614.2232572847 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Perf_Convert.ToDateTime_String(value: "February 26, 2009")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -11.185781620720178 (T) = (0 -4773.054294656579) / Math.Sqrt((6803.577714594896 / (8)) + (1674.393109590004 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.10653043911956098 = (4313.531852277269 - 4773.054294656579) / 4313.531852277269 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Perf_Convert.ToDateTime_String(value: "12/12/1999")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -8.898226617379379 (T) = (0 -4061.9867160407516) / Math.Sqrt((1281.8475338133496 / (8)) + (3979.8656345787404 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.11306299234177743 = (3649.3772086472145 - 4061.9867160407516) / 3649.3772086472145 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6aaaaaa5c7b1160a237bb381074e707c3ea1e9b4
Compare d0fcd6206daac01f6279adc51e40d7e1df4fed19
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Format_OneArg - Duration of single invocation 22.94 μs 26.11 μs 1.14 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Format_OneArg(s: "Testing {0}, {0:C}, {0:E} - {0:F4}{0:G}{0:N} , !!", o: 3.14159)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -14.754105009614797 (T) = (0 -25919.25934616703) / Math.Sqrt((75399.71845663093 / (8)) + (71040.21536659935 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.13724098570534327 = (22791.35176445589 - 25919.25934616703) / 22791.35176445589 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline 6aaaaaa5c7b1160a237bb381074e707c3ea1e9b4
Compare d0fcd6206daac01f6279adc51e40d7e1df4fed19
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetDateTime - Duration of single invocation 115.44 μs 135.56 μs 1.17 0.04 False
GetDateTimeOffset - Duration of single invocation 157.11 μs 184.63 μs 1.18 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Get.GetDateTime

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/3/2022 11:38:23 AM, 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -28.72249480249754 (T) = (0 -135226.94915053635) / Math.Sqrt((4739875.677878952 / (8)) + (217335.372457 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.2163298251795453 = (111176.21746270604 - 135226.94915053635) / 111176.21746270604 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Get.GetDateTimeOffset

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -16.34050021701175 (T) = (0 -183016.41436944253) / Math.Sqrt((1988533.0422251758 / (8)) + (5223187.314061253 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.1778531871347916 = (155381.34664698108 - 183016.41436944253) / 155381.34664698108 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6aaaaaa5c7b1160a237bb381074e707c3ea1e9b4
Compare d0fcd6206daac01f6279adc51e40d7e1df4fed19
Diff Diff

Regressions in System.Globalization.Tests.Perf_DateTimeCultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 7.19 μs 7.69 μs 1.07 0.01 False
ToStringHebrewIsrael - Duration of single invocation 6.06 μs 7.48 μs 1.23 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_DateTimeCultureInfo*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring: ja)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -18.934295593990772 (T) = (0 -7689.9540769058385) / Math.Sqrt((7483.829436271553 / (8)) + (38.29787405922673 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.08233902524693706 = (7104.940224391674 - 7689.9540769058385) / 7104.940224391674 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToStringHebrewIsrael

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -20.800853271272786 (T) = (0 -7417.80151888841) / Math.Sqrt((3906.027986627568 / (8)) + (8141.807648774115 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.2335666626218093 = (6013.296033084012 - 7417.80151888841) / 6013.296033084012 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6aaaaaa5c7b1160a237bb381074e707c3ea1e9b4
Compare d0fcd6206daac01f6279adc51e40d7e1df4fed19
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Ctor - Duration of single invocation 176.31 μs 214.49 μs 1.22 0.04 False
CtorInvoke - Duration of single invocation 177.55 μs 217.59 μs 1.23 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Ctor(Options: IgnoreCase, Compiled)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -16.103684369274024 (T) = (0 -212466.10585045663) / Math.Sqrt((7422551.30784735 / (8)) + (8179003.745182776 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.20449150291410678 = (176394.85653192503 - 212466.10585045663) / 176394.85653192503 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.CtorInvoke(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -12.040894535510347 (T) = (0 -214568.64407141518) / Math.Sqrt((2328174.53510613 / (8)) + (18293417.66085801 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.20830683312389311 = (177577.94476480834 - 214568.64407141518) / 177577.94476480834 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6aaaaaa5c7b1160a237bb381074e707c3ea1e9b4
Compare d0fcd6206daac01f6279adc51e40d7e1df4fed19
Diff Diff

Regressions in System.Tests.Perf_DateTime

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ParseR - Duration of single invocation 978.14 ns 1.24 μs 1.27 0.04 False
ParseO - Duration of single invocation 1.01 μs 1.28 μs 1.26 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTime*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_DateTime.ParseR

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -73.23224231353876 (T) = (0 -1241.5876884230343) / Math.Sqrt((74.73680437307347 / (8)) + (7.593229666452117 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.2719359533194779 = (976.1400998082952 - 1241.5876884230343) / 976.1400998082952 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_DateTime.ParseO

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -20.125403793271357 (T) = (0 -1264.1233252217507) / Math.Sqrt((285.4711001615497 / (8)) + (250.65876380699243 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.25315812206370525 = (1008.7500555316898 - 1264.1233252217507) / 1008.7500555316898 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6aaaaaa5c7b1160a237bb381074e707c3ea1e9b4
Compare d0fcd6206daac01f6279adc51e40d7e1df4fed19
Diff Diff

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 2.20 μs 2.54 μs 1.15 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.Parse(numberString: 123)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -3.8950495082301946 (T) = (0 -2474.6407549840806) / Math.Sqrt((1506.1215461844793 / (8)) + (8970.78987336206 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.12057905795491732 = (2208.358917130182 - 2474.6407549840806) / 2208.358917130182 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6aaaaaa5c7b1160a237bb381074e707c3ea1e9b4
Compare d0fcd6206daac01f6279adc51e40d7e1df4fed19
Diff Diff

Regressions in System.Tests.Perf_Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 3.48 μs 4.44 μs 1.28 0.04 False
ToStringWithFormat - Duration of single invocation 2.74 μs 3.73 μs 1.36 0.02 False
TryParse - Duration of single invocation 2.07 μs 2.20 μs 1.06 0.01 False
ToStringWithFormat - Duration of single invocation 3.74 μs 4.65 μs 1.25 0.01 False
ToStringWithCultureInfo - Duration of single invocation 3.37 μs 4.39 μs 1.30 0.03 False
ToStringWithFormat - Duration of single invocation 3.41 μs 4.43 μs 1.30 0.03 False
ToStringWithCultureInfo - Duration of single invocation 3.33 μs 4.32 μs 1.30 0.03 False
Parse - Duration of single invocation 2.04 μs 2.19 μs 1.07 0.01 False
ToStringWithFormat - Duration of single invocation 3.71 μs 4.68 μs 1.26 0.01 False
ToStringWithFormat - Duration of single invocation 3.40 μs 4.40 μs 1.30 0.03 False
TryParse - Duration of single invocation 2.09 μs 2.25 μs 1.07 0.03 False
ToStringWithFormat - Duration of single invocation 2.73 μs 3.76 μs 1.38 0.01 False
ToString - Duration of single invocation 2.60 μs 3.40 μs 1.31 0.03 False
ToStringWithFormat - Duration of single invocation 3.37 μs 4.39 μs 1.30 0.03 False
ToString - Duration of single invocation 3.31 μs 4.33 μs 1.31 0.05 False
ToStringWithCultureInfo - Duration of single invocation 2.63 μs 3.41 μs 1.30 0.03 False
Parse - Duration of single invocation 2.08 μs 2.25 μs 1.08 0.03 False
ToStringWithFormat - Duration of single invocation 2.70 μs 3.72 μs 1.38 0.02 False
ToStringWithFormat - Duration of single invocation 2.68 μs 3.48 μs 1.30 0.01 False
ToStringWithFormat - Duration of single invocation 3.41 μs 4.43 μs 1.30 0.03 False
ToStringWithFormat - Duration of single invocation 3.56 μs 4.41 μs 1.24 0.01 False
ToStringWithFormat - Duration of single invocation 2.64 μs 3.44 μs 1.30 0.01 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Single.ToString(value: -3.4028235E+38)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -17.543742356809066 (T) = (0 -4382.995432323265) / Math.Sqrt((3475.5409161068555 / (8)) + (6097.517843521993 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.30929978759068505 = (3347.587369878565 - 4382.995432323265) / 3347.587369878565 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -124.49923473132118 (T) = (0 -3731.274324490489) / Math.Sqrt((499.58826096137466 / (8)) + (8.426024317502774 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.37442902494116515 = (2714.7813795988577 - 3731.274324490489) / 2714.7813795988577 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.TryParse(value: "3.4028235E+38")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/3/2023 12:19:11 AM, 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -15.675661168854191 (T) = (0 -2194.6686798884) / Math.Sqrt((926.9440346038299 / (8)) + (39.10075724053822 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.09065302979650046 = (2012.251944413424 - 2194.6686798884) / 2012.251944413424 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G17")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -58.341564837713165 (T) = (0 -4659.681561567347) / Math.Sqrt((2074.9669690041624 / (8)) + (77.02466378104273 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.27566488151092383 = (3652.7473861695744 - 4659.681561567347) / 3652.7473861695744 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithCultureInfo(value: -3.4028235E+38, culture: zh)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -25.70328737761175 (T) = (0 -4348.133767710518) / Math.Sqrt((951.3551285294039 / (8)) + (2871.6345060943513 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.3039253463525053 = (3334.649318592996 - 4348.133767710518) / 3334.649318592996 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "R")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -76.28271994213405 (T) = (0 -4422.484677693932) / Math.Sqrt((805.0811097547299 / (8)) + (161.46883309540647 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.30258563957308116 = (3395.1584781353718 - 4422.484677693932) / 3395.1584781353718 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithCultureInfo(value: 3.4028235E+38, culture: zh)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -42.33178075274954 (T) = (0 -4343.896585517656) / Math.Sqrt((795.1672340908356 / (8)) + (1052.8127956077788 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.32237393380443874 = (3284.923027044527 - 4343.896585517656) / 3284.923027044527 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.Parse(value: "3.4028235E+38")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/3/2023 12:19:11 AM, 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -13.989894726496235 (T) = (0 -2196.252609456889) / Math.Sqrt((874.1484564197004 / (8)) + (149.35367668386533 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.09456207959468606 = (2006.5126048128366 - 2196.252609456889) / 2006.5126048128366 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G17")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -73.70165930978469 (T) = (0 -4683.079023573131) / Math.Sqrt((1382.4008386076907 / (8)) + (15.873318891176693 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.2683551289918405 = (3692.2458990610176 - 4683.079023573131) / 3692.2458990610176 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "R")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -36.80001781303915 (T) = (0 -4374.892821130712) / Math.Sqrt((740.4682420608405 / (8)) + (1350.5253583571616 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.30392212557907433 = (3355.1795274490137 - 4374.892821130712) / 3355.1795274490137 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.TryParse(value: "-3.4028235E+38")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/3/2023 12:19:11 AM, 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -24.306708714414434 (T) = (0 -2243.9135077300916) / Math.Sqrt((477.90087776655383 / (8)) + (4.308274050850127 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.09315781119016019 = (2052.6894513858547 - 2243.9135077300916) / 2052.6894513858547 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -93.27437412351823 (T) = (0 -3753.158243444487) / Math.Sqrt((704.9889615058904 / (8)) + (67.71999461662384 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.3783283795703894 = (2722.9782822975085 - 3753.158243444487) / 2722.9782822975085 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToString(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -50.97978552963666 (T) = (0 -3398.3014224847207) / Math.Sqrt((1937.8357824120135 / (8)) + (9.397434833498092 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.30844423879062133 = (2597.2076774366237 - 3398.3014224847207) / 2597.2076774366237 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -31.81176930288309 (T) = (0 -4359.228497279181) / Math.Sqrt((1290.5632052362246 / (8)) + (1745.5088443127433 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.3066227625746794 = (3336.2563565702703 - 4359.228497279181) / 3336.2563565702703 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToString(value: 3.4028235E+38)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -45.79563111525225 (T) = (0 -4306.947720862883) / Math.Sqrt((1188.1685724561926 / (8)) + (692.4161681521749 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.3097642862666733 = (3288.3380361052014 - 4306.947720862883) / 3288.3380361052014 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithCultureInfo(value: 12345, culture: zh)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -19.044778405158716 (T) = (0 -3454.0614325222414) / Math.Sqrt((1560.1100140216465 / (8)) + (3667.918191417214 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.3304265004806067 = (2596.206127339231 - 3454.0614325222414) / 2596.206127339231 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.Parse(value: "-3.4028235E+38")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/3/2023 12:19:11 AM, 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -17.799196314710073 (T) = (0 -2240.420811396176) / Math.Sqrt((731.2419793046471 / (8)) + (46.76639556291572 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.0930367768331601 = (2049.721344132002 - 2240.420811396176) / 2049.721344132002 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -85.02370472561049 (T) = (0 -3714.552731066068) / Math.Sqrt((412.177697992909 / (8)) + (197.06133125577537 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.38963305948441695 = (2673.045740898137 - 3714.552731066068) / 2673.045740898137 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "R")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -34.922324179684665 (T) = (0 -3460.592497085433) / Math.Sqrt((587.4473570110903 / (8)) + (928.2549061639866 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.3054367844924576 = (2650.9077560817173 - 3460.592497085433) / 2650.9077560817173 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -37.33137416699049 (T) = (0 -4406.564691557513) / Math.Sqrt((866.3364375245445 / (8)) + (1293.9861599878582 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.30348327130386726 = (3380.60701550059 - 4406.564691557513) / 3380.60701550059 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "G17")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -26.205603897295305 (T) = (0 -4380.8567697664475) / Math.Sqrt((804.531933434685 / (8)) + (1908.2674849797945 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.24110634194268243 = (3529.799680911442 - 4380.8567697664475) / 3529.799680911442 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "G")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -43.62998339502826 (T) = (0 -3426.284848771057) / Math.Sqrt((1763.185019458285 / (8)) + (197.69910584757747 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.29453705785386575 = (2646.725968935402 - 3426.284848771057) / 2646.725968935402 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline 6aaaaaa5c7b1160a237bb381074e707c3ea1e9b4
Compare d0fcd6206daac01f6279adc51e40d7e1df4fed19
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Doubles

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteDoubles - Duration of single invocation 536.89 ms 629.37 ms 1.17 0.03 False
WriteDoubles - Duration of single invocation 521.48 ms 614.84 ms 1.18 0.03 False
WriteDoubles - Duration of single invocation 519.08 ms 618.90 ms 1.19 0.02 False
WriteDoubles - Duration of single invocation 536.97 ms 627.23 ms 1.17 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Doubles*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: True, SkipValidation: True)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -24.351648292811852 (T) = (0 -625783441.8666667) / Math.Sqrt((24640145551589.973 / (8)) + (25657896689275.656 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.183728927624847 = (528654345.82417583 - 625783441.8666667) / 528654345.82417583 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: False, SkipValidation: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -36.43821199634554 (T) = (0 -612599001.9190476) / Math.Sqrt((13018249483810.857 / (8)) + (10032452582738.113 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.1810742467397103 = (518679501.8264881 - 612599001.9190476) / 518679501.8264881 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: False, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -15.764608605424147 (T) = (0 -612880407.4761904) / Math.Sqrt((16022736977309.684 / (8)) + (72588988208250.19 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.18931687112387266 = (515321376.797619 - 612880407.4761904) / 515321376.797619 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: True, SkipValidation: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -30.351744602836874 (T) = (0 -624512304.3076923) / Math.Sqrt((16379766255524.547 / (8)) + (14764417096968 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.1754224172146376 = (531308825.7986264 - 624512304.3076923) / 531308825.7986264 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6aaaaaa5c7b1160a237bb381074e707c3ea1e9b4
Compare d0fcd6206daac01f6279adc51e40d7e1df4fed19
Diff Diff

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseDouble - Duration of single invocation 1.81 μs 2.02 μs 1.11 0.04 False
TryParseSingle - Duration of single invocation 1.58 μs 1.77 μs 1.12 0.01 False
TryParseSingle - Duration of single invocation 1.60 μs 1.79 μs 1.12 0.03 False
TryParseDouble - Duration of single invocation 1.80 μs 1.99 μs 1.11 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseDouble(value: -1.7976931348623157e+308)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -20.61320374817579 (T) = (0 -2006.7011887914373) / Math.Sqrt((238.25187303574594 / (8)) + (145.89439556322625 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.11621330158162335 = (1797.7757351108728 - 2006.7011887914373) / 1797.7757351108728 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: 3.4028235E+38)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -42.4757073143543 (T) = (0 -1769.1943491575348) / Math.Sqrt((211.76787716678228 / (8)) + (0.18912430515778447 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.14121911779459762 = (1550.26700970143 - 1769.1943491575348) / 1550.26700970143 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: -3.4028235E+38)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -15.43775746011091 (T) = (0 -1778.7384035915845) / Math.Sqrt((408.78060922517017 / (8)) + (254.96244748602834 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.13119748596181266 = (1572.4384341954167 - 1778.7384035915845) / 1572.4384341954167 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseDouble(value: 1.7976931348623157e+308)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -29.822732411853607 (T) = (0 -1987.7141066302925) / Math.Sqrt((263.2316644404067 / (8)) + (22.482496874481114 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.11072390868450961 = (1789.566327949535 - 1987.7141066302925) / 1789.566327949535 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6aaaaaa5c7b1160a237bb381074e707c3ea1e9b4
Compare d0fcd6206daac01f6279adc51e40d7e1df4fed19
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromReader - Duration of single invocation 4.08 μs 4.37 μs 1.07 0.04 False
DeserializeFromReader - Duration of single invocation 4.06 μs 4.38 μs 1.08 0.06 False
DeserializeFromString - Duration of single invocation 3.74 μs 4.04 μs 1.08 0.02 False
DeserializeFromUtf8Bytes - Duration of single invocation 2.72 μs 3.00 μs 1.10 0.08 False
DeserializeFromString - Duration of single invocation 3.77 μs 4.08 μs 1.08 0.04 False
DeserializeFromUtf8Bytes - Duration of single invocation 2.72 μs 3.04 μs 1.12 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromReader(Mode: Reflection)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -4.46891336808839 (T) = (0 -4459.993450802553) / Math.Sqrt((1192.8313482632398 / (8)) + (16011.932078474782 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.09948834094272742 = (4056.426325520149 - 4459.993450802553) / 4056.426325520149 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;.DeserializeFromReader(Mode: SourceGen)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/19/2022 10:34:07 PM, 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -7.461191171954259 (T) = (0 -4356.39228094526) / Math.Sqrt((6880.998083992911 / (8)) + (746.283338219315 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.06399549959000769 = (4094.370965501189 - 4356.39228094526) / 4094.370965501189 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;.DeserializeFromString(Mode: Reflection)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -9.676356815766201 (T) = (0 -4015.1317007334533) / Math.Sqrt((713.1473905764614 / (8)) + (1361.2767908218555 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.07165577772108833 = (3746.6617399028673 - 4015.1317007334533) / 3746.6617399028673 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;.DeserializeFromUtf8Bytes(Mode: Reflection)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -7.627132623752042 (T) = (0 -2968.5555341996037) / Math.Sqrt((292.9709874074078 / (8)) + (2288.8647234070363 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.09684964862652913 = (2706.4379679720164 - 2968.5555341996037) / 2706.4379679720164 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;.DeserializeFromString(Mode: SourceGen)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/18/2022 12:43:31 PM, 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -5.5819086530438815 (T) = (0 -4030.9508376223516) / Math.Sqrt((682.6343527528022 / (8)) + (4281.690078233206 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.06990358549990956 = (3767.583259139094 - 4030.9508376223516) / 3767.583259139094 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;.DeserializeFromUtf8Bytes(Mode: SourceGen)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -9.956604071797164 (T) = (0 -3011.4962026107505) / Math.Sqrt((667.3182872628744 / (8)) + (1491.236530460897 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.10521070292470752 = (2724.8163582215225 - 3011.4962026107505) / 2724.8163582215225 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline 6aaaaaa5c7b1160a237bb381074e707c3ea1e9b4
Compare d0fcd6206daac01f6279adc51e40d7e1df4fed19
Diff Diff

Regressions in System.Text.Tests.Perf_StringBuilder

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Append_Primitives - Duration of single invocation 23.23 μs 27.34 μs 1.18 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_StringBuilder.Append_Primitives

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -24.075144259662675 (T) = (0 -27181.853413533696) / Math.Sqrt((31792.35957567034 / (8)) + (50693.00081922098 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.1787749921446883 = (23059.407940168847 - 27181.853413533696) / 23059.407940168847 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6aaaaaa5c7b1160a237bb381074e707c3ea1e9b4
Compare d0fcd6206daac01f6279adc51e40d7e1df4fed19
Diff Diff

Regressions in System.Tests.Perf_Decimal

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Round - Duration of single invocation 121.24 ns 320.43 ns 2.64 0.20 False
Divide - Duration of single invocation 934.91 ns 1.72 μs 1.84 0.01 False
Floor - Duration of single invocation 109.72 ns 312.07 ns 2.84 0.34 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Decimal*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Decimal.Round

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -123.54604666362525 (T) = (0 -319.33008926387095) / Math.Sqrt((12.573832787650987 / (8)) + (2.4082941719845694 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -1.813716376311549 = (113.49050385898352 - 319.33008926387095) / 113.49050385898352 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Decimal.Divide

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -64.42526839391823 (T) = (0 -1712.2396679026865) / Math.Sqrt((68.04802934193587 / (8)) + (276.855812321028 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.8385470949492195 = (931.3004124868384 - 1712.2396679026865) / 931.3004124868384 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Decimal.Floor

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -92.17701900564963 (T) = (0 -312.2518512925923) / Math.Sqrt((41.26440655398193 / (8)) + (0.06463795424492866 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -2.0537888625495833 = (102.25063530813188 - 312.2518512925923) / 102.25063530813188 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6aaaaaa5c7b1160a237bb381074e707c3ea1e9b4
Compare d0fcd6206daac01f6279adc51e40d7e1df4fed19
Diff Diff

Regressions in System.Globalization.Tests.Perf_NumberCultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 3.02 μs 3.95 μs 1.31 0.04 False
ToString - Duration of single invocation 3.06 μs 3.84 μs 1.25 0.03 False
ToString - Duration of single invocation 3.08 μs 3.85 μs 1.25 0.03 False
ToString - Duration of single invocation 3.02 μs 3.82 μs 1.27 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_NumberCultureInfo*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: fr)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -12.492319939516143 (T) = (0 -3879.1551640064417) / Math.Sqrt((407.4758187539677 / (8)) + (9867.478490656451 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.2942728155874288 = (2997.169620877665 - 3879.1551640064417) / 2997.169620877665 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: da)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/3/2022 11:38:23 AM, 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -56.18919959554155 (T) = (0 -3829.3801133341694) / Math.Sqrt((1065.8184838331501 / (8)) + (161.8028166506448 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.2734225774250018 = (3007.1558186737902 - 3829.3801133341694) / 3007.1558186737902 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: )

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -26.565184697133102 (T) = (0 -3823.4597104366453) / Math.Sqrt((2430.6373210713036 / (8)) + (1266.0336666031656 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.27010305882250185 = (3010.3539109506055 - 3823.4597104366453) / 3010.3539109506055 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: ja)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -46.77661932887904 (T) = (0 -3808.8058959641066) / Math.Sqrt((569.3858380692321 / (8)) + (451.14907343813144 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.26832745632960886 = (3003.014621307927 - 3808.8058959641066) / 3003.014621307927 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6aaaaaa5c7b1160a237bb381074e707c3ea1e9b4
Compare d0fcd6206daac01f6279adc51e40d7e1df4fed19
Diff Diff

Regressions in System.Xml.Tests.Perf_XmlConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DateTime_ToString_Unspecified - Duration of single invocation 1.26 μs 1.49 μs 1.19 0.03 False
DateTime_ToString_RoundtripKind - Duration of single invocation 1.28 μs 1.45 μs 1.13 0.06 False
DateTime_ToString - Duration of single invocation 1.33 μs 1.58 μs 1.19 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Tests.Perf_XmlConvert*'

Payloads

Baseline
Compare

Histogram

System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_Unspecified

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/8/2022 2:00:38 AM, 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -15.292390765706996 (T) = (0 -1476.7522630486242) / Math.Sqrt((322.1640327594666 / (8)) + (348.121305881363 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.17869472674866121 = (1252.8708490298686 - 1476.7522630486242) / 1252.8708490298686 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_RoundtripKind

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/8/2022 2:00:38 AM, 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -9.635929303562989 (T) = (0 -1430.5778260012075) / Math.Sqrt((809.5435152186802 / (8)) + (761.621166358012 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.1735425913052608 = (1219.025058485574 - 1430.5778260012075) / 1219.025058485574 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Xml.Tests.Perf_XmlConvert.DateTime_ToString

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 12/8/2022 2:00:38 AM, 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -42.91405770880396 (T) = (0 -1574.0418075103203) / Math.Sqrt((72.80159134603888 / (8)) + (53.15743679497287 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.19452979971162399 = (1317.7082797685882 - 1574.0418075103203) / 1317.7082797685882 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6aaaaaa5c7b1160a237bb381074e707c3ea1e9b4
Compare d0fcd6206daac01f6279adc51e40d7e1df4fed19
Diff Diff

Regressions in LinqBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Where01LinqQueryX - Duration of single invocation 6.07 secs 9.86 secs 1.62 0.03 False
Where01LinqMethodX - Duration of single invocation 6.16 secs 9.77 secs 1.59 0.04 False
Where01ForX - Duration of single invocation 5.79 secs 9.69 secs 1.67 0.06 False
Where01LinqMethodNestedX - Duration of single invocation 6.72 secs 10.59 secs 1.57 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'

Payloads

Baseline
Compare

Histogram

LinqBenchmarks.Where01LinqQueryX

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -43.80312396745148 (T) = (0 -9776745949.987179) / Math.Sqrt((8696884724322866 / (8)) + (12536481574337402 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.6240324402233179 = (6020043508.88631 - 9776745949.987179) / 6020043508.88631 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### LinqBenchmarks.Where01LinqMethodX

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -71.01028972496611 (T) = (0 -9740851951.066666) / Math.Sqrt((14381025933614912 / (8)) + (1887137427040019.5 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.6172741894964252 = (6023005878.860714 - 9740851951.066666) / 6023005878.860714 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### LinqBenchmarks.Where01ForX

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -76.25608508610756 (T) = (0 -9645394793.766666) / Math.Sqrt((6749382155659189 / (8)) + (3540174732525294 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.6783921175626896 = (5746806537.541071 - 9645394793.766666) / 5746806537.541071 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### LinqBenchmarks.Where01LinqMethodNestedX

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -25.61540290580423 (T) = (0 -10443230975.3) / Math.Sqrt((8624347642434833 / (8)) + (41388789946804800 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.5672164651210597 = (6663553636.474404 - 10443230975.3) / 6663553636.474404 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 6aaaaaa5c7b1160a237bb381074e707c3ea1e9b4
Compare d0fcd6206daac01f6279adc51e40d7e1df4fed19
Diff Diff

Regressions in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FormatterDouble - Duration of single invocation 5.10 μs 5.91 μs 1.16 0.03 False
FormatterDouble - Duration of single invocation 3.10 μs 3.89 μs 1.26 0.06 False
FormatterDouble - Duration of single invocation 5.19 μs 5.85 μs 1.13 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: -1.7976931348623157E+308)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -17.288508611880484 (T) = (0 -5866.102998343095) / Math.Sqrt((4483.585042494223 / (8)) + (3061.3800283382498 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.15576453061176224 = (5075.517411178976 - 5866.102998343095) / 5075.517411178976 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -22.920501542991627 (T) = (0 -3865.53546118999) / Math.Sqrt((4848.582006488845 / (8)) + (1243.1448161172762 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.2622353394290553 = (3062.4522546948187 - 3865.53546118999) / 3062.4522546948187 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: 1.7976931348623157E+308)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 1/13/2023 1:32:26 AM, 1/17/2023 12:02:27 AM falls between 1/7/2023 8:48:01 PM and 1/17/2023 12:02:27 AM.
IsRegressionStdDev: Marked as regression because -13.230189344350741 (T) = (0 -5913.159327524668) / Math.Sqrt((6882.357622290797 / (8)) + (7225.956140812513 / (2))) is less than -2.3060041352041716 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (2) - 2, .025) and -0.17597799471377965 = (5028.290796345953 - 5913.159327524668) / 5028.290796345953 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler performanceautofiler bot added ampere arm64 PGO Applied if there were any profile guided optimization updates in the observed interval. untriaged labels Jan 17, 2023
@kotlarmilos
Copy link
Member

Duplicate of #11790.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
ampere arch-arm64 branch-refs/heads/main kind-micro_mono mono-interpreter os-linux perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-mono
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kotlarmilos