Run Information
Regressions in System.Collections.TryGetValueFalse<Int32, Int32>
_2.png)
_3.png)
Test Report
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueFalse<Int32, Int32>*'
Payloads
Baseline
Compare
Histogram
Edge Detector Info
Collection Data
System.Collections.TryGetValueFalse<Int32, Int32>.ConcurrentDictionary(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.5190243849206353 > 2.4036435081910237.
IsChangePoint: Marked as a change because one of 8/6/2022 4:25:34 AM, 8/15/2022 9:13:40 PM, 9/13/2022 12:23:36 PM, 9/24/2022 7:43:48 AM, 9/27/2022 3:32:36 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 3:32:36 AM.
IsRegressionStdDev: Marked as regression because -18.96761416807978 (T) = (0 -2531.2749920005836) / Math.Sqrt((4762.9302636739385 / (45)) + (102.93410430935235 / (13))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (13) - 2, .025) and -0.0868651462502898 = (2328.968778448312 - 2531.2749920005836) / 2328.968778448312 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.TryGetValueFalse<Int32, Int32>.IDictionary(Size: 512)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.189232560069029 > 4.154752209147254.
IsChangePoint: Marked as a change because one of 9/23/2022 1:16:35 AM, 9/27/2022 3:32:36 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 3:32:36 AM.
IsRegressionStdDev: Marked as regression because -14.445665294315434 (T) = (0 -5399.457485776981) / Math.Sqrt((7800.283818254304 / (38)) + (188851.75236255076 / (20))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (20) - 2, .025) and -0.3564589621258156 = (3980.553512149795 - 5399.457485776981) / 3980.553512149795 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.TryGetValueFalse<Int32, Int32>.Dictionary(Size: 512)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.312845970588235 > 4.10624376084124.
IsChangePoint: Marked as a change because one of 8/23/2022 11:46:46 PM, 9/1/2022 8:04:48 PM, 9/13/2022 12:23:36 PM, 9/24/2022 1:05:07 PM, 9/27/2022 3:32:36 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 3:32:36 AM.
IsRegressionStdDev: Marked as regression because -21.186422586815983 (T) = (0 -5116.308816871219) / Math.Sqrt((36197.789434224804 / (46)) + (24833.31452200058 / (12))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (12) - 2, .025) and -0.28421284578723527 = (3984.003768265432 - 5116.308816871219) / 3984.003768265432 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
I got a preview 4 build through the pipeline this afternoon. Profiles should get picked up by maestro’s next pr into runtime. Let me know if you see anything that looks wrong.
I’m still working on this. I’m getting very close to having it running on .NET 8.0 preview 2. There’s an issue that makes preview 3 and newer fail on ARM. I’m working on a fix for that also, but current priority is to get us up to date with at least preview 2.
Yes, I have tracked down the issue to the fact that when we build the shared framework for the PGO instrumented SDK when the ASP.NET part of the shared framework is deployed it also deploys a copy of the NetCore.App part of the shared framework. When this happens the specially instrumented binaries that were produced are clobbered. I am working on getting a fix in to stop this.
I had noticed this now that I am back from paternity leave. I will get working on fixing this.