It looks like this change caused a regression in AOT, caused by something in this date range:
https://github.com/dotnet/runtime/compare/cd2f0ffb287251c26f03850d1a596abafb99d5c7...c6f5267686688aeabaa84aeb02efc2b411e7d64b. I’d have to guess this: https://github.com/dotnet/runtime/commit/c7c6aa032175f4ee48c38757d1d9f32df25f02c6 Some of these are as big as a 2.5x slowdown.
@vargaz Anything to do here?
Automated issue below.
Run Information
Regressions in System.Collections.CtorDefaultSize<String>
Test Report
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorDefaultSize<String>*'
Payloads
Baseline
Compare
Histogram
Edge Detector Info
Collection Data
System.Collections.CtorDefaultSize<String>.SortedDictionary
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.73335573461183 > 48.6412374911.
IsChangePoint: Marked as a change because one of 9/29/2022 8:01:59 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -23.5769379262111 (T) = (0 -51.80269873824366) / Math.Sqrt((1.2727628493292753 / (41)) + (0.6847383195274711 / (22))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (22) - 2, .025) and -0.12800522805184716 = (45.92416546482764 - 51.80269873824366) / 45.92416546482764 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.CtorDefaultSize<String>.SortedSet
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.24752981648574 > 16.6085654509353.
IsChangePoint: Marked as a change because one of 9/29/2022 8:01:59 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -48.30175899355235 (T) = (0 -24.649499605313142) / Math.Sqrt((0.4730085769754052 / (41)) + (0.3864414471602412 / (23))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (23) - 2, .025) and -0.49224978292076427 = (16.518346919821255 - 24.649499605313142) / 16.518346919821255 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.CtorDefaultSize<String>.HashSet
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.56570063802273 > 27.435984642843305.
IsChangePoint: Marked as a change because one of 8/31/2022 4:31:58 AM, 9/29/2022 8:01:59 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -28.75717212338262 (T) = (0 -33.11827583088849) / Math.Sqrt((1.630402644977107 / (41)) + (0.2220099249245795 / (20))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (20) - 2, .025) and -0.24352847904772937 = (26.632502905160518 - 33.11827583088849) / 26.632502905160518 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.CtorDefaultSize<String>.Dictionary
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.63335110324571 > 29.044955892172425.
IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 9/29/2022 8:01:59 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -29.108112186065227 (T) = (0 -34.04146925640556) / Math.Sqrt((1.1992188762513298 / (41)) + (0.34683607802698657 / (23))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (23) - 2, .025) and -0.2195600031637879 = (27.912910531745084 - 34.04146925640556) / 27.912910531745084 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.CtorDefaultSize<String>.SortedList
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.20238573083015 > 26.60454029441908.
IsChangePoint: Marked as a change because one of 9/29/2022 8:01:59 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -12.417590466118261 (T) = (0 -33.797764771554974) / Math.Sqrt((0.49049493267061556 / (41)) + (10.81239729348717 / (24))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (24) - 2, .025) and -0.3330834924041096 = (25.353074255389213 - 33.797764771554974) / 25.353074255389213 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
### Run Information
Regressions in System.Collections.CtorDefaultSize<Int32>
Test Report
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CtorDefaultSize<Int32>*'
Payloads
Baseline
Compare
Histogram
Edge Detector Info
Collection Data
System.Collections.CtorDefaultSize<Int32>.SortedDictionary
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.7550601931671 > 35.92533690650893.
IsChangePoint: Marked as a change because one of 9/29/2022 8:01:59 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -14.846143817519133 (T) = (0 -38.998995419953246) / Math.Sqrt((1.6992966167429813 / (41)) + (1.1880614958087041 / (24))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (24) - 2, .025) and -0.12969390482113838 = (34.52173659919663 - 38.998995419953246) / 34.52173659919663 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Collections.IterateForEach<String>
Test Report
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach<String>*'
Payloads
Baseline
Compare
Histogram
Edge Detector Info
Collection Data
System.Collections.IterateForEach<String>.ImmutableArray(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.153798066850448 > 1.6803698052925844.
IsChangePoint: Marked as a change because one of 9/2/2022 11:09:40 AM, 9/10/2022 11:12:52 PM, 9/21/2022 9:54:19 PM, 9/24/2022 1:05:07 PM, 9/29/2022 8:01:59 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -7.624334426765053 (T) = (0 -1950.5284241196825) / Math.Sqrt((32025.0676492788 / (41)) + (6575.741920437325 / (24))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (24) - 2, .025) and -0.14543331385229122 = (1702.8738386870525 - 1950.5284241196825) / 1702.8738386870525 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Tests.Perf_Int32
Test Report
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int32*'
Payloads
Baseline
Compare
Histogram
Edge Detector Info
Collection Data
System.Tests.Perf_Int32.ToString(value: 2147483647)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 61.1820935606927 > 55.93182035291981.
IsChangePoint: Marked as a change because one of 9/29/2022 8:01:59 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -16.301454043987736 (T) = (0 -58.464037286556575) / Math.Sqrt((1.3204787355337462 / (41)) + (1.372112835183621 / (22))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (22) - 2, .025) and -0.09379104481688806 = (53.45082825791837 - 58.464037286556575) / 53.45082825791837 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int32.ToStringHex(value: -2147483648)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.86829579541057 > 45.81260976273628.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 8/31/2022 10:21:55 AM, 9/13/2022 6:36:50 PM, 9/17/2022 6:06:25 PM, 9/21/2022 9:54:19 PM, 9/29/2022 7:42:19 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -32.04462026944698 (T) = (0 -47.70958590025842) / Math.Sqrt((0.1857251458030036 / (38)) + (0.3059255391232149 / (25))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (25) - 2, .025) and -0.09636368429499821 = (43.51620414254912 - 47.70958590025842) / 43.51620414254912 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int32.ToString(value: 12345)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.904393685109135 > 32.89836825847774.
IsChangePoint: Marked as a change because one of 8/25/2022 2:41:52 PM, 9/29/2022 8:01:59 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -14.937873187574798 (T) = (0 -37.47420888664807) / Math.Sqrt((0.6222430606742206 / (41)) + (2.7554508611198227 / (22))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (22) - 2, .025) and -0.1756060321827595 = (31.87650272350962 - 37.47420888664807) / 31.87650272350962 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int32.TryParse(value: "-2147483648")
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 45.382370158644285 > 40.38091807389151.
IsChangePoint: Marked as a change because one of 7/21/2022 3:55:55 PM, 7/27/2022 9:27:06 AM, 8/2/2022 6:54:34 PM, 9/6/2022 5:23:23 PM, 9/24/2022 1:05:07 PM, 9/29/2022 7:42:19 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -11.908169526863801 (T) = (0 -42.967410381740066) / Math.Sqrt((0.8676118537215057 / (38)) + (1.4530560078252297 / (25))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (25) - 2, .025) and -0.08560458065866881 = (39.57924565468437 - 42.967410381740066) / 39.57924565468437 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int32.TryParseSpan(value: "2147483647")
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 46.63276022795546 > 41.06790051263776.
IsChangePoint: Marked as a change because one of 7/21/2022 3:55:55 PM, 8/24/2022 2:12:57 PM, 9/6/2022 5:23:23 PM, 9/29/2022 7:42:19 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -16.581763370193425 (T) = (0 -43.75776629611237) / Math.Sqrt((0.364497176583538 / (38)) + (1.480629968675883 / (26))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (26) - 2, .025) and -0.10833971821800119 = (39.48046395600305 - 43.75776629611237) / 39.48046395600305 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int32.TryParseSpan(value: "12345")
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.70270837989453 > 31.595365748747042.
IsChangePoint: Marked as a change because one of 9/29/2022 8:01:59 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -22.985524429706565 (T) = (0 -34.080243375211) / Math.Sqrt((0.22213998313461794 / (41)) + (0.4308788095003647 / (21))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (21) - 2, .025) and -0.12185415947792652 = (30.378497140012215 - 34.080243375211) / 30.378497140012215 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int32.ToStringHex(value: 2147483647)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.33762801770216 > 47.5917133183744.
IsChangePoint: Marked as a change because one of 8/17/2022 9:10:20 PM, 8/31/2022 10:21:55 AM, 9/29/2022 8:01:59 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -21.87736837624892 (T) = (0 -48.26914185302055) / Math.Sqrt((0.5008594954877544 / (41)) + (0.10463414626407662 / (23))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (23) - 2, .025) and -0.06234444461205551 = (45.43643269170327 - 48.26914185302055) / 45.43643269170327 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int32.TryParseSpan(value: "-2147483648")
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 46.98472945250976 > 41.47091047773013.
IsChangePoint: Marked as a change because one of 7/21/2022 3:55:55 PM, 7/27/2022 9:27:06 AM, 8/2/2022 6:54:34 PM, 9/6/2022 5:23:23 PM, 9/29/2022 7:42:19 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -13.027566593168164 (T) = (0 -44.07192935676823) / Math.Sqrt((0.7021577794196064 / (38)) + (1.4275125338618861 / (25))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (25) - 2, .025) and -0.08845233934209593 = (40.49045398111512 - 44.07192935676823) / 40.49045398111512 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int32.TryParseSpan(value: "4")
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.3372737768588 > 21.731724396648538.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/29/2022 8:01:59 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -31.94175414537656 (T) = (0 -26.146040840526393) / Math.Sqrt((0.44619206511913634 / (41)) + (0.15778719605848188 / (22))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (22) - 2, .025) and -0.19639190304599333 = (21.854077057826135 - 26.146040840526393) / 21.854077057826135 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int32.TryParse(value: "12345")
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.52241286351984 > 31.693600314997866.
IsChangePoint: Marked as a change because one of 7/21/2022 3:55:55 PM, 7/27/2022 4:00:59 PM, 8/2/2022 6:54:34 PM, 9/6/2022 5:23:23 PM, 9/29/2022 8:01:59 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -5.957225614345597 (T) = (0 -32.97157420513012) / Math.Sqrt((0.1855417853115592 / (41)) + (3.918472671889059 / (23))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (23) - 2, .025) and -0.08173588974175053 = (30.480244316384496 - 32.97157420513012) / 30.480244316384496 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int32.TryParse(value: "4")
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.61862701491588 > 22.43761858577799.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/2/2022 11:09:40 AM, 9/29/2022 2:35:34 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -29.464286059937336 (T) = (0 -25.518379397010264) / Math.Sqrt((0.2808064375085862 / (39)) + (0.22023630980835945 / (26))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (26) - 2, .025) and -0.16896184644562712 = (21.829950630640383 - 25.518379397010264) / 21.829950630640383 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int32.ToString(value: -2147483648)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 71.64792029510802 > 61.92640418091867.
IsChangePoint: Marked as a change because one of 8/2/2022 6:54:34 PM, 9/21/2022 9:54:19 PM, 9/29/2022 8:01:59 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -20.633731110813173 (T) = (0 -67.87142791775793) / Math.Sqrt((1.0831548056589655 / (41)) + (3.291077236492413 / (21))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (21) - 2, .025) and -0.14955773271915102 = (59.04133910457512 - 67.87142791775793) / 59.04133910457512 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Closing this old issue. The charts seemed to show random spikes. There are other newer issues per preview, which we can focus on.