runtime: .net 7.0 perf regression with stopwatch

Same code runs about 15% faster in older .net 6.0 than in 7.0 tested on Windows 10 and net 7.0.10 runtime

          int[] arrayToSort = new int[1000];
          Random random = new Random();
          for (int i = 0; i < arrayToSort.Length; i++)
          {
              arrayToSort[i] = random.Next();
          }

          stopwatch.Reset();
          stopwatch.Start();
          for (int i = 0; i < 500000; i++)
          {
              Array.Sort(arrayToSort);
          }
          stopwatch.Stop();
          totalElapsedTime += stopwatch.ElapsedMilliseconds;
          Console.WriteLine($"Sorting took: {stopwatch.ElapsedMilliseconds} ms");

About this issue

Original URL
State: closed
Created a year ago
Comments: 17 (8 by maintainers)

Most upvoted comments

@johnnygiter is there is anything we can do for you here aside from the explanations above?

Almost all is clear in practice just all, it can be closed, thank you!

Ok, I will close. Feel free to re-open or create new issues if you see other issues with performance. Thanks!

AndyAyersMS on Aug 14, 2023

I just don’t know how many similar cases like this stopwatch exist maybe some other “slowdowns” or maybe “just” this?

This is not slowdown, since randomization changes the result. This is just unstable result of badly-written benchmark.

LOL, unfortunately, I see that you still don’t understand the problem, that it’s not about any randomization but about stopwatch 😃 so root cause is that line stopwatch = new Stopwatch();

Likely what is happening is that one extra line changes the memory address where the array is allocated slightly and this is what is leading to the performance impact you’re seeing. Try adding two or more new Stopwatch and you may see performance change yet again.

See eg https://github.com/dotnet/BenchmarkDotNet/issues/1513

If you’re interested learning more about the ways today’s microarchtectures can make benchmarking tricky you might consider reading about stabilizer.

AndyAyersMS on Aug 14, 2023

I checked your benchmark both with stopwatch = new Stopwatch(); line

commented:

Method	Job	Runtime	Mean	Error	StdDev	Allocated
SortBenchmark	.NET 6.0	.NET 6.0	1.357 s	0.0167 s	0.0157 s	5568 B
SortBenchmark	.NET 7.0	.NET 7.0	1.379 s	0.0233 s	0.0218 s	600 B
SortBenchmark	.NET 8.0	.NET 8.0	1.314 s	0.0120 s	0.0112 s	400 B

and uncommented:

Method	Job	Runtime	Mean	Error	StdDev	Allocated
SortBenchmark	.NET 6.0	.NET 6.0	1.362 s	0.0263 s	0.0271 s	640 B
SortBenchmark	.NET 7.0	.NET 7.0	1.668 s	0.0140 s	0.0131 s	600 B
SortBenchmark	.NET 8.0	.NET 8.0	1.326 s	0.0245 s	0.0229 s	400 B

It seems like there was some problem in .NET 7 (maybe OSR?), but in the latest .NET 8 preview 7 it is fixed

KeterSCP on Aug 14, 2023