Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Regressions in Exceptions.Handling #84308

Closed
performanceautofiler bot opened this issue Apr 4, 2023 · 5 comments · Fixed by #90033
Closed

Regressions in Exceptions.Handling #84308

performanceautofiler bot opened this issue Apr 4, 2023 · 5 comments · Fixed by #90033
Labels
arch-x64 area-VM-coreclr os-linux Linux OS (any supported distro) runtime-coreclr specific to the CoreCLR runtime tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark
Milestone
8.0.0

Comments

@performanceautofiler
Copy link

No description provided.

@performanceautofiler performanceautofiler bot added arch-x64 os-linux Linux OS (any supported distro) runtime-coreclr specific to the CoreCLR runtime untriaged New issue has not been triaged by the area owner labels Apr 4, 2023
@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline f561a058b13523c4b045803c9876e823b4cfe0fc
Compare ce4857970dff1e987a3991238c8b32e8eae3cab7
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Exceptions.Handling

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ThrowAndCatchDeepRecursive - Duration of single invocation 15.48 μs 18.03 μs 1.17 0.01 False
CatchAndThrowOtherDeep - Duration of single invocation 199.44 μs 219.65 μs 1.10 0.01 False
ThrowAndCatch - Duration of single invocation 29.10 μs 31.03 μs 1.07 0.01 False
MultipleNestedTryCatch_LastCatches - Duration of single invocation 6.40 μs 6.79 μs 1.06 0.06 False
MultipleNestedTryFinally - Duration of single invocation 6.41 μs 7.09 μs 1.11 0.03 False
CatchAndRethrowDeep - Duration of single invocation 190.29 μs 206.09 μs 1.08 0.00 True
ThrowAndCatch - Duration of single invocation 30.02 μs 32.24 μs 1.07 0.00 True
CatchAndRethrowDeep - Duration of single invocation 194.65 μs 211.77 μs 1.09 0.01 False
ThrowAndCatchDeep - Duration of single invocation 38.13 μs 41.91 μs 1.10 0.00 True
MultipleNestedTryCatch_FirstCatches - Duration of single invocation 6.31 μs 6.86 μs 1.09 0.04 False
ThrowAndCatch_ManyCatchBlocks - Duration of single invocation 29.44 μs 32.23 μs 1.09 0.02 False
ThrowAndCatchDeep - Duration of single invocation 14.69 μs 16.89 μs 1.15 0.01 False
ThrowAndCatchWhenFinally - Duration of single invocation 8.93 μs 9.49 μs 1.06 0.01 True
ThrowAndCatchDeepRecursive - Duration of single invocation 39.27 μs 43.71 μs 1.11 0.01 True
ThrowAndCatch_ManyCatchBlocks - Duration of single invocation 29.78 μs 32.00 μs 1.07 0.01 True
ThrowAndCatchDeepRecursive - Duration of single invocation 18.52 μs 20.69 μs 1.12 0.00 True
ThrowAndCatchDeepRecursive - Duration of single invocation 38.26 μs 42.54 μs 1.11 0.01 False
TryAndFinallyDeep - Duration of single invocation 18.56 μs 20.66 μs 1.11 0.01 True
TryAndFinallyDeep - Duration of single invocation 16.17 μs 18.21 μs 1.13 0.01 False
ThrowAndCatchWhen - Duration of single invocation 6.41 μs 7.04 μs 1.10 0.07 False
ThrowAndCatchDeep - Duration of single invocation 17.58 μs 19.66 μs 1.12 0.00 True
ThrowAndCatch_ManyCatchBlocks - Duration of single invocation 6.52 μs 7.19 μs 1.10 0.04 False
CatchAndThrowOtherDeep - Duration of single invocation 194.06 μs 214.85 μs 1.11 0.01 True
ThrowAndCatch_ManyCatchBlocks - Duration of single invocation 9.07 μs 9.66 μs 1.07 0.01 True
MultipleNestedTryCatch_FirstCatches - Duration of single invocation 8.70 μs 9.32 μs 1.07 0.00 True
ThrowAndCatchWhen - Duration of single invocation 8.67 μs 9.51 μs 1.10 0.00 True
ThrowAndCatch - Duration of single invocation 8.83 μs 9.55 μs 1.08 0.01 True
ThrowAndCatchFinally - Duration of single invocation 6.19 μs 6.98 μs 1.13 0.05 False
ThrowAndCatchWhenFinally - Duration of single invocation 6.14 μs 7.00 μs 1.14 0.06 False
TryAndCatchDeep_CaugtAtTheTop - Duration of single invocation 15.80 μs 18.17 μs 1.15 0.02 False
ThrowAndCatchFinally - Duration of single invocation 8.84 μs 9.36 μs 1.06 0.01 True
ThrowAndCatchDeep - Duration of single invocation 37.98 μs 41.82 μs 1.10 0.01 False
TryAndCatchDeep_CaugtAtTheTop - Duration of single invocation 18.55 μs 20.48 μs 1.10 0.00 True

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Exceptions.Handling*'

Payloads

Baseline
Compare

Histogram

Exceptions.Handling.ThrowAndCatchDeepRecursive(kind: Hardware)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.033538856772548 > 16.26590125300773.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 3/28/2023 12:51:33 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -4.983386667753461 (T) = (0 -18107.201909296335) / Math.Sqrt((45250854.573263064 / (30)) + (29184.414432298316 / (35))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (35) - 2, .025) and -0.5108037368418138 = (11985.145037533237 - 18107.201909296335) / 11985.145037533237 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Exceptions.Handling.CatchAndThrowOtherDeep(kind: Hardware)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 219.65319468896712 > 210.47084078448592.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 3/28/2023 12:51:33 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -4.124525894188253 (T) = (0 -219152.82069018536) / Math.Sqrt((7472455816.195268 / (30)) + (4042705.7591395867 / (35))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (35) - 2, .025) and -0.42267199879428785 = (154043.11104451132 - 219152.82069018536) / 154043.11104451132 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Exceptions.Handling.ThrowAndCatch(kind: ReflectionHardware)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.026296760645383 > 30.549974304286874.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 3/28/2023 12:51:33 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -4.073306793674217 (T) = (0 -31655.62271732413) / Math.Sqrt((157048726.20392105 / (30)) + (135026.4541166516 / (35))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (35) - 2, .025) and -0.4174719264272136 = (22332.45126562274 - 31655.62271732413) / 22332.45126562274 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Exceptions.Handling.MultipleNestedTryCatch_LastCatches(kind: Hardware)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.790685131834408 > 6.747281928139415.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 3/28/2023 12:51:33 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -4.172022084679639 (T) = (0 -7084.292174702221) / Math.Sqrt((7761392.139298306 / (30)) + (17542.87619478009 / (35))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (35) - 2, .025) and -0.4282309461862126 = (4960.186721635824 - 7084.292174702221) / 4960.186721635824 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Exceptions.Handling.MultipleNestedTryFinally(kind: Hardware)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.085922955848255 > 6.7320489729329855.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 3/28/2023 12:51:33 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -4.320271489518469 (T) = (0 -7070.984102275607) / Math.Sqrt((7568222.140568847 / (30)) + (12816.534468589894 / (34))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (34) - 2, .025) and -0.44322722098384193 = (4899.425398486696 - 7070.984102275607) / 4899.425398486696 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Exceptions.Handling.CatchAndRethrowDeep(kind: Software)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 206.09270397727275 > 198.8533619540663.
IsChangePoint: Marked as a change because one of 3/4/2023 1:07:20 AM, 3/27/2023 3:21:55 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -78.64475273784944 (T) = (0 -206200.98523826088) / Math.Sqrt((404741.60884120414 / (23)) + (1195053.5721380098 / (41))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (41) - 2, .025) and -0.08987159269354725 = (189197.50420198444 - 206200.98523826088) / 189197.50420198444 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Exceptions.Handling.ThrowAndCatch(kind: ReflectionSoftware)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.23842710062241 > 31.1831802856985.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -38.16850448586281 (T) = (0 -31994.003674308082) / Math.Sqrt((45971.066374686525 / (23)) + (56168.473064827434 / (42))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (42) - 2, .025) and -0.07400508280399988 = (29789.4341345001 - 31994.003674308082) / 29789.4341345001 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Exceptions.Handling.CatchAndRethrowDeep(kind: Hardware)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 211.7736002533784 > 205.04523854882814.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 3/28/2023 12:51:33 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -4.0739128116778796 (T) = (0 -212824.7150973708) / Math.Sqrt((7099960197.220132 / (30)) + (1107389.9761917244 / (34))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (34) - 2, .025) and -0.41743659917735854 = (150147.60816878048 - 212824.7150973708) / 150147.60816878048 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Exceptions.Handling.ThrowAndCatchDeep(kind: ReflectionSoftware)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.90826626624046 > 40.23328394943226.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -41.55348407853975 (T) = (0 -42191.00624409911) / Math.Sqrt((63411.91514245333 / (23)) + (223877.05374960753 / (42))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (42) - 2, .025) and -0.09717858590618482 = (38454.09196466644 - 42191.00624409911) / 38454.09196466644 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Exceptions.Handling.MultipleNestedTryCatch_FirstCatches(kind: Hardware)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.860840970056145 > 6.618340591615613.
IsChangePoint: Marked as a change because one of 3/27/2023 6:14:52 PM, 3/28/2023 12:51:33 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -3.9463681840259155 (T) = (0 -6894.737558226836) / Math.Sqrt((6596432.129279415 / (30)) + (14014.420551113457 / (33))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (33) - 2, .025) and -0.36734124334781965 = (5042.441008614392 - 6894.737558226836) / 5042.441008614392 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Exceptions.Handling.ThrowAndCatch_ManyCatchBlocks(kind: ReflectionHardware)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.22728392588635 > 30.820725159801132.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 3/28/2023 12:51:33 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -4.000499403356067 (T) = (0 -31850.780973416928) / Math.Sqrt((160716675.23088193 / (30)) + (105103.3493345686 / (35))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (35) - 2, .025) and -0.4100278336293517 = (22588.760458319728 - 31850.780973416928) / 22588.760458319728 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Exceptions.Handling.ThrowAndCatchDeep(kind: Hardware)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.893536433537964 > 15.465644644993375.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 3/28/2023 12:51:33 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -4.8892380350507665 (T) = (0 -17087.076674767228) / Math.Sqrt((40817832.24141375 / (30)) + (22543.08735154454 / (34))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (34) - 2, .025) and -0.5011498310974533 = (11382.659026298055 - 17087.076674767228) / 11382.659026298055 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Exceptions.Handling.ThrowAndCatchWhenFinally(kind: Software)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.48826869051192 > 9.257177150306083.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -30.446069673218457 (T) = (0 -9505.530118808332) / Math.Sqrt((7428.96551559084 / (23)) + (7405.389804491558 / (41))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (41) - 2, .025) and -0.07744637203182986 = (8822.276788479938 - 9505.530118808332) / 8822.276788479938 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Exceptions.Handling.ThrowAndCatchDeepRecursive(kind: ReflectionSoftware)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 43.71137641679201 > 41.390150545463335.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -39.744103854759345 (T) = (0 -43324.46641776555) / Math.Sqrt((77498.20718690756 / (23)) + (250022.82467050402 / (41))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (41) - 2, .025) and -0.09800887413880505 = (39457.30079071171 - 43324.46641776555) / 39457.30079071171 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Exceptions.Handling.ThrowAndCatch_ManyCatchBlocks(kind: ReflectionSoftware)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.00269204312115 > 31.389879910376838.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -36.61417126886463 (T) = (0 -32187.205309053283) / Math.Sqrt((24517.101116893245 / (23)) + (97089.39908322714 / (40))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (40) - 2, .025) and -0.07207818960003624 = (30023.188253705142 - 32187.205309053283) / 30023.188253705142 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Exceptions.Handling.ThrowAndCatchDeepRecursive(kind: Software)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.69175685042449 > 19.39078137774979.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -55.78924042047397 (T) = (0 -20625.306064478453) / Math.Sqrt((33023.2162594831 / (23)) + (11110.06462771384 / (42))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (42) - 2, .025) and -0.12553799922675984 = (18324.842056552472 - 20625.306064478453) / 18324.842056552472 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Exceptions.Handling.ThrowAndCatchDeepRecursive(kind: ReflectionHardware)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 42.53708214529 > 40.69534866627522.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 3/28/2023 12:51:33 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -4.37027868719663 (T) = (0 -43085.78824747006) / Math.Sqrt((278840972.16104877 / (30)) + (274088.7714200317 / (34))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (34) - 2, .025) and -0.4479575184333267 = (29756.251615785237 - 43085.78824747006) / 29756.251615785237 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Exceptions.Handling.TryAndFinallyDeep(kind: Software)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.662319015604247 > 19.550242781851765.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -53.008204400238846 (T) = (0 -20783.75893642632) / Math.Sqrt((32109.09879548522 / (23)) + (20071.934553132553 / (42))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (42) - 2, .025) and -0.12411004661021575 = (18489.078537373014 - 20783.75893642632) / 18489.078537373014 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Exceptions.Handling.TryAndFinallyDeep(kind: Hardware)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.209184669649844 > 16.625931161251838.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 3/28/2023 12:51:33 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -4.923416051955526 (T) = (0 -18271.178691986257) / Math.Sqrt((46452945.00634649 / (30)) + (35036.815143557775 / (35))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (35) - 2, .025) and -0.5047051172185731 = (12142.69725204383 - 18271.178691986257) / 12142.69725204383 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Exceptions.Handling.ThrowAndCatchWhen(kind: Hardware)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.039350922230091 > 6.536197613154784.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 3/28/2023 12:51:33 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -4.48401798784183 (T) = (0 -6971.472694558446) / Math.Sqrt((7192573.7303162925 / (30)) + (12710.116405053548 / (35))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (35) - 2, .025) and -0.460228992898987 = (4774.232485767871 - 6971.472694558446) / 4774.232485767871 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Exceptions.Handling.ThrowAndCatchDeep(kind: Software)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.65595258815551 > 18.3280203055762.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -60.31643290931944 (T) = (0 -19634.846614998798) / Math.Sqrt((25046.80359960628 / (23)) + (9046.089051025581 / (41))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (41) - 2, .025) and -0.12507274447915417 = (17452.06850965769 - 19634.846614998798) / 17452.06850965769 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Exceptions.Handling.ThrowAndCatch_ManyCatchBlocks(kind: Hardware)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.1879115197189485 > 6.838348618122104.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 3/28/2023 12:51:33 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -4.232223437475321 (T) = (0 -7121.51143040019) / Math.Sqrt((7770188.143158777 / (30)) + (12231.842601356693 / (34))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (34) - 2, .025) and -0.43401774506633145 = (4966.125039178493 - 7121.51143040019) / 4966.125039178493 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Exceptions.Handling.CatchAndThrowOtherDeep(kind: Software)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 214.847272566047 > 203.6125810300202.
IsChangePoint: Marked as a change because one of 3/3/2023 11:00:07 PM, 3/27/2023 3:21:55 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -36.507898295529685 (T) = (0 -210814.0165197564) / Math.Sqrt((1362215.5104701107 / (23)) + (6370039.783583214 / (41))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (41) - 2, .025) and -0.08721935974633767 = (193902.00756629466 - 210814.0165197564) / 193902.00756629466 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Exceptions.Handling.ThrowAndCatch_ManyCatchBlocks(kind: Software)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.663575868207714 > 9.470599802579056.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -26.678088308418086 (T) = (0 -9712.450808136475) / Math.Sqrt((4140.47789404408 / (23)) + (20981.38140351111 / (41))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (41) - 2, .025) and -0.07787007104924239 = (9010.780676637567 - 9712.450808136475) / 9010.780676637567 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Exceptions.Handling.MultipleNestedTryCatch_FirstCatches(kind: Software)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.322730024363562 > 9.163020594827387.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -28.317511246183006 (T) = (0 -9456.017152144233) / Math.Sqrt((6678.62685006289 / (23)) + (11742.082292295734 / (41))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (41) - 2, .025) and -0.07749283474541817 = (8775.94434711803 - 9456.017152144233) / 8775.94434711803 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Exceptions.Handling.ThrowAndCatchWhen(kind: Software)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.509472584777443 > 9.249301402112055.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -30.127698187055017 (T) = (0 -9478.490156490174) / Math.Sqrt((5766.216141310185 / (23)) + (9262.014456623197 / (41))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (41) - 2, .025) and -0.07456593150622329 = (8820.761833761226 - 9478.490156490174) / 8820.761833761226 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Exceptions.Handling.ThrowAndCatch(kind: Software)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.551841889171433 > 9.242851369715202.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -28.524529493119235 (T) = (0 -9462.42380747293) / Math.Sqrt((9452.875597234655 / (23)) + (7050.117403046723 / (41))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (41) - 2, .025) and -0.07849644164529604 = (8773.718152502724 - 9462.42380747293) / 8773.718152502724 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Exceptions.Handling.ThrowAndCatchFinally(kind: Hardware)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.977237529312589 > 6.588675356037553.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 3/28/2023 12:51:33 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -4.385052502081273 (T) = (0 -6927.337631798273) / Math.Sqrt((7201995.772166328 / (30)) + (10547.315011468125 / (34))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (34) - 2, .025) and -0.4500152531003544 = (4777.423973290326 - 6927.337631798273) / 4777.423973290326 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Exceptions.Handling.ThrowAndCatchWhenFinally(kind: Hardware)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.998152154910914 > 6.589369044742465.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 3/28/2023 12:51:33 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -4.41638327819623 (T) = (0 -6965.052585439521) / Math.Sqrt((7244251.567069175 / (30)) + (8059.960676318256 / (33))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (33) - 2, .025) and -0.4529476065845923 = (4793.7396736638675 - 6965.052585439521) / 4793.7396736638675 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Exceptions.Handling.TryAndCatchDeep_CaugtAtTheTop(kind: Hardware)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.17293127172653 > 16.57393092936665.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 3/28/2023 12:51:33 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -4.823731086318639 (T) = (0 -18152.306897270988) / Math.Sqrt((46483608.14994266 / (30)) + (21792.89771072034 / (34))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (34) - 2, .025) and -0.4944319127156387 = (12146.626917438572 - 18152.306897270988) / 12146.626917438572 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Exceptions.Handling.ThrowAndCatchFinally(kind: Software)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.364175546672643 > 9.24155912012909.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -30.907948256599123 (T) = (0 -9499.11733939278) / Math.Sqrt((7602.03936378216 / (23)) + (7156.35546511203 / (41))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (41) - 2, .025) and -0.07889340348684254 = (8804.500341454377 - 9499.11733939278) / 8804.500341454377 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Exceptions.Handling.ThrowAndCatchDeep(kind: ReflectionHardware)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.81977290788969 > 39.776169143953766.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 3/28/2023 12:51:33 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -4.27768155734632 (T) = (0 -41847.89119589174) / Math.Sqrt((266526988.94298342 / (30)) + (252382.54873631036 / (34))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (34) - 2, .025) and -0.43845130025598983 = (29092.323937865953 - 41847.89119589174) / 29092.323937865953 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Exceptions.Handling.TryAndCatchDeep_CaugtAtTheTop(kind: Software)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.480180592660293 > 19.478895212078314.
IsChangePoint: Marked as a change because one of 3/27/2023 3:21:55 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsRegressionStdDev: Marked as regression because -53.68331141773125 (T) = (0 -20553.29892886329) / Math.Sqrt((27255.21613402702 / (23)) + (14270.968085185314 / (41))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (41) - 2, .025) and -0.11391837724012203 = (18451.350968629107 - 20553.29892886329) / 18451.350968629107 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@EgorBo EgorBo changed the title [Perf] Linux/x64: 33 Regressions on 3/28/2023 4:36:21 PM Regressions in Exceptions.Handling Apr 4, 2023
@EgorBo EgorBo transferred this issue from dotnet/perf-autofiling-issues Apr 4, 2023
@dotnet-issue-labeler dotnet-issue-labeler bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Apr 4, 2023
@EgorBo EgorBo added tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark area-VM-coreclr and removed needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners labels Apr 4, 2023
@EgorBo
Copy link
Member

EgorBo commented Apr 4, 2023 •
edited
Loading

Regressed in #83784 cc @tannergooding

@EgorBo
Copy link
Member

EgorBo commented Apr 4, 2023 •
edited
Loading

More:

@mangod9
Copy link
Member

mangod9 commented Jul 24, 2023

@tannergooding, is this something you plan to look into for 8?

@mangod9 mangod9 removed the untriaged New issue has not been triaged by the area owner label Jul 24, 2023
@mangod9 mangod9 added this to the 8.0.0 milestone Jul 24, 2023
@janvorli
Copy link
Member

janvorli commented Aug 4, 2023

I've done an experiment - adding custom copy constructor and assignment operator to the CONTEXT for amd64 and skipping copying of the ymm high / zmm registers based on the ContextFlags and XStateFeaturesMask of the source context. I can see 3..8% gain in the exception handling tests.

janvorli added a commit to janvorli/runtime that referenced this issue Aug 4, 2023
@janvorli
Improve EH performance on Unix amd64
2453a3e 
Few months ago, a change to enable AVX512 on Unix was merged in.
That change has enlarged the CONTEXT structure significantly and
it was found that it has caused 6..17% regressions in exception handling
microbenchmarks.

This change gets some of the lost performance back by adding custom
copy constructor and assignment operator to the CONTEXT structure
and preventing copying of the AVX / AVX512 stuff if the source
context doesn't have it. I have observed 3..9% gain on my machine.

Close dotnet#84308
@janvorli janvorli mentioned this issue Aug 4, 2023
Merged
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Aug 4, 2023
janvorli added a commit that referenced this issue Aug 6, 2023
@janvorli
Recover some lost EH performance on Unix amd64 (#90033)
1018da5 
* Improve EH performance on Unix amd64

Few months ago, a change to enable AVX512 on Unix was merged in.
That change has enlarged the CONTEXT structure significantly and
it was found that it has caused 6..17% regressions in exception handling
microbenchmarks.

This change gets some of the lost performance back by adding custom
copy constructor and assignment operator to the CONTEXT structure
and preventing copying of the AVX / AVX512 stuff if the source
context doesn't have it. I have observed 3..9% gain on my machine.

Close #84308

* Fix gcc build and size for non-xstate case

* Fix yet another gcc break

* Fix build break

* Revert the gcc fixes and disable the warning

The zeroing of CONTEXT structure is used at too many places and
it is valid, so rather than trying to modify all the places by
using placement new, just disable the warning for gcc.
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Aug 6, 2023
@ghost ghost locked as resolved and limited conversation to collaborators Sep 5, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
arch-x64 area-VM-coreclr os-linux Linux OS (any supported distro) runtime-coreclr specific to the CoreCLR runtime tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark
Projects
None yet
Milestone
8.0.0
Development

Successfully merging a pull request may close this issue.

Recover some lost EH performance on Unix amd64 janvorli/runtime
4 participants
@EgorBo @AndyAyersMS @janvorli @mangod9