runtime: [Perf] Linux/x64: 16 Regressions on 9/22/2022 3:16:12 AM

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare	b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff	Diff

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Queue - Duration of single invocation	969.31 μs	1.10 ms	1.13	0.07	False
ImmutableArray - Duration of single invocation	996.73 μs	1.11 ms	1.12	0.05	False
ICollection - Duration of single invocation	967.59 μs	1.06 ms	1.10	0.05	False
List - Duration of single invocation	954.73 μs	1.06 ms	1.11	0.06	False
Array - Duration of single invocation	974.99 μs	1.07 ms	1.10	0.06	False
Stack - Duration of single invocation	1.00 ms	1.24 ms	1.23	0.06	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsTrue<Int32>.Queue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.095966200892857 > 1010.798569984976.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -26.205242060142314 (T) = (0 -1082176.5961467223) / Math.Sqrt((134557800.542629 / (33)) + (378753722.89546967 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11677096832909976 = (969022.8586134029 - 1082176.5961467223) / 969022.8586134029 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1139794651785715 > 1.060484347207031.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -27.791974606900073 (T) = (0 -1115382.3246755877) / Math.Sqrt((180256514.8127183 / (33)) + (235012253.979763 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.10483220306009597 = (1009549.0714212264 - 1115382.3246755877) / 1009549.0714212264 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.ICollection(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0617160416666667 > 1018.8446187011718.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -16.056018293701424 (T) = (0 -1087710.4426824627) / Math.Sqrt((185033592.69745567 / (33)) + (1162193806.342326 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11694145311763579 = (973829.4157195234 - 1087710.4426824627) / 973829.4157195234 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.List(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0611418856944443 > 1018.2846668750002.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -35.482093782068404 (T) = (0 -1074002.2283184354) / Math.Sqrt((69967824.32398583 / (33)) + (179800464.4794185 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11025724102303357 = (967345.3940537317 - 1074002.2283184354) / 967345.3940537317 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Array(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0702191419642857 > 1023.6990125195313.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -30.120564854575687 (T) = (0 -1086000.919079096) / Math.Sqrt((291634961.6495431 / (33)) + (86090478.21488371 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.10701630982952783 = (981016.1868765348 - 1086000.919079096) / 981016.1868765348 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;Int32&gt;.Stack(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2352883976648354 > 1024.4168104882813.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -60.858087292552526 (T) = (0 -1236941.1502558445) / Math.Sqrt((337610251.7351471 / (33)) + (203005793.13338426 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.2641636960719056 = (978465.9645735368 - 1236941.1502558445) / 978465.9645735368 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare	b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff	Diff

Regressions in System.Memory.SequenceReader

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TryReadTo - Duration of single invocation	4.81 μs	5.38 μs	1.12	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.SequenceReader*'

Payloads

Baseline Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.SequenceReader.TryReadTo

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.378923124141483 > 5.053520181621476.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -51.38447919574725 (T) = (0 -5367.488921550735) / Math.Sqrt((2937.7007674825045 / (32)) + (472.35950356068855 / (26))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (26) - 2, .025) and -0.11159539603981007 = (4828.635437563927 - 5367.488921550735) / 4828.635437563927 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare	b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff	Diff

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Stack - Duration of single invocation	1.89 ms	2.38 ms	1.26	0.06	False
ImmutableArray - Duration of single invocation	1.93 ms	2.14 ms	1.11	0.03	False
Array - Duration of single invocation	1.90 ms	2.11 ms	1.11	0.03	False
List - Duration of single invocation	1.89 ms	2.10 ms	1.11	0.05	False
Queue - Duration of single invocation	1.89 ms	2.23 ms	1.18	0.06	False
ICollection - Duration of single invocation	1.87 ms	2.10 ms	1.12	0.05	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline Compare

Histogram

Edge Detector Info

Collection Data

System.Collections.ContainsFalse<Int32>.Stack(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3842154708333334 > 1.9866159997796478.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -63.97195320199993 (T) = (0 -2394094.2410446103) / Math.Sqrt((1695724547.8362103 / (33)) + (190835740.57041073 / (25))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (25) - 2, .025) and -0.2583031159565782 = (1902637.1394022887 - 2394094.2410446103) / 1902637.1394022887 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1394566171875 > 2.0256352184960935.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -31.077029944721016 (T) = (0 -2151332.1590478206) / Math.Sqrt((360162614.46956015 / (33)) + (997867569.4428535 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11286796194600314 = (1933142.3247066252 - 2151332.1590478206) / 1933142.3247066252 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Array(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1087881843750003 > 1.9809466521310766.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -45.259498036275815 (T) = (0 -2113783.74757238) / Math.Sqrt((353745729.22009766 / (33)) + (319361059.55712956 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11444507736727372 = (1896714.1499390076 - 2113783.74757238) / 1896714.1499390076 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.List(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1026052706473215 > 1.9608499316145835.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -68.80938666641624 (T) = (0 -2099614.6035013716) / Math.Sqrt((211620946.8627083 / (33)) + (92498111.32462098 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11542629051873408 = (1882342.7611025164 - 2099614.6035013716) / 1882342.7611025164 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.Queue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2304079871651785 > 2.008788108359375.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -23.59913129326589 (T) = (0 -2147578.562214543) / Math.Sqrt((1116474284.4023762 / (33)) + (1587306697.1320746 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11986933064211758 = (1917704.5959309835 - 2147578.562214543) / 1917704.5959309835 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.ICollection(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.0979382890625 > 1.9806727393359376.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -42.85148744252998 (T) = (0 -2103699.1899045617) / Math.Sqrt((347858606.2090312 / (33)) + (396049417.258107 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.11533974964320046 = (1886151.005178054 - 2103699.1899045617) / 1886151.005178054 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare	b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff	Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Contains_ElementNotFound - Duration of single invocation	832.17 ns	895.38 ns	1.08	0.03	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline Compare

Histogram

Edge Detector Info

Collection Data

System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input: ICollection)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 895.3778104156972 > 849.9629668568364.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -29.21081282954959 (T) = (0 -892.9335262280066) / Math.Sqrt((168.6694102042165 / (33)) + (82.54061160444941 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.10395462812604744 = (808.8498417219853 - 892.9335262280066) / 808.8498417219853 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	34c522b7e9eaa9bddb6d93ba471cf1c7d4ff3266
Compare	b5b2af07114a7e94c82676ddd23ff51af1bed0d5
Diff	Diff

Regressions in System.Memory.Span<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
LastIndexOfValue - Duration of single invocation	1.87 μs	2.49 μs	1.33	0.06	False
IndexOfValue - Duration of single invocation	1.98 μs	2.11 μs	1.07	0.05	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline Compare

Histogram

Edge Detector Info

Collection Data

System.Memory.Span<Int32>.LastIndexOfValue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.486075723145284 > 1.961551330630418.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -32.74534469892047 (T) = (0 -2378.1927078714825) / Math.Sqrt((666.1354551089895 / (33)) + (5810.079235983002 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.2737630503998014 = (1867.0605236390152 - 2378.1927078714825) / 1867.0605236390152 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.IndexOfValue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1059525350629564 > 1.9711226404728943.
IsChangePoint: Marked as a change because one of 8/5/2022 10:47:59 PM, 9/21/2022 9:54:19 PM, 9/27/2022 7:01:57 AM falls between 9/18/2022 2:31:46 PM and 9/27/2022 7:01:57 AM.
IsRegressionStdDev: Marked as regression because -26.75366215584882 (T) = (0 -2108.271736496125) / Math.Sqrt((824.5871390603775 / (33)) + (1462.8924148464291 / (26))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (26) - 2, .025) and -0.1291609019154783 = (1867.1136530849672 - 2108.271736496125) / 1867.1136530849672 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 29 (7 by maintainers)

Commits related to this issue

use 7.0.0 Microsoft.Extensions* when targeting net7.0, so .NET 6 vs 7 diff shows the actual difference for Extensions benchmarks, fixes https://github.com/dotnet/runtime/issues/77900 — committed to adamsitnik/performance by adamsitnik 2 years ago

Most upvoted comments

Sure enough, this benchmark no longer shows up as regressed in the GA vs GA report.

Benchmark.GetChildKeysTests.AddChainedConfigurationEmpty

Result	Ratio	Alloc Delta	Operating System	Bit	Processor Name
Faster	1.66	-887829	Windows 11	Arm64	Microsoft SQ1 3.0 GHz
Faster	1.34	-887772	macOS 13.0	Arm64	Apple M1
Same	0.98	-887744	Windows 10	X64	Intel Xeon CPU E5-1650 v4 3.60GHz
Same	1.09	-887796	Windows 10	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)
Faster	1.66	-887817	Windows 10	X64	Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)
Same	1.05	-887751	Windows 11	X64	AMD Ryzen Threadripper PRO 3945WX 12-Cores
Faster	1.20	-887762	Windows 11	X64	11th Gen Intel Core i9-11900 2.50GHz
Same	1.02	-888033	ubuntu 18.04	X64	Intel Xeon CPU E5-1650 v4 3.60GHz
Faster	1.64	-444254	raspbian 11	Arm	ARMv7 Processor rev 3 (v7l)
Same	1.02	-887773	macOS Monterey 12.6	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)

dakersnar on Dec 1, 2022