runtime: [MONO][PERF] Perf Regression in System.Tests.Perf_String
My best guess is this regression is caused by https://github.com/dotnet/runtime/commit/9bb0427e765e8aac89be0147599bd173486682ea @stephentoub does that sound plausible?
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 3cbbadee12cc95bd62c70786d5408a2277a21e0a |
| Compare | 2740fbbe0b3bf2e5b4aea9a121047b81b233a5cc |
| Diff | Diff |
Regressions in System.Collections.ContainsFalse<String>
| Benchmark | Baseline | Test | Test/Base | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|
| HashSet | 16.25 μs | 17.74 μs | 1.09 |
_1.png)
_2.png)
_3.png)
_4.png)
_5.png)
_6.png)
_7.png)
Historical Data in Reporting System
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Collections.ContainsFalse<String>*'
Payloads
Histogram
System.Collections.ContainsFalse<String>.HashSet(Size: 512)
Docs
Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 3cbbadee12cc95bd62c70786d5408a2277a21e0a |
| Compare | 2740fbbe0b3bf2e5b4aea9a121047b81b233a5cc |
| Diff | Diff |
Regressions in System.Text.Json.Tests.Utf8JsonReaderCommentsTests
| Benchmark | Baseline | Test | Test/Base | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|
| Utf8JsonReaderCommentParsing | 19.48 μs | 20.59 μs | 1.06 |
Historical Data in Reporting System
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'
Payloads
Histogram
System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: LongMultiLine)
There are performance regressions in several System.Tests.Perf_String benchmarks on mono. My best guess is that they are due to this change: https://github.com/dotnet/runtime/commit/9bb0427e765e8aac89be0147599bd173486682ea. @stephentoub Is that plausible?
The autofiled issue is below; this is from the interpreter, but we see similar regressions in JIT mode.
Docs
Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 19 (19 by maintainers)
Agree with Steve’s initial assessment at https://github.com/dotnet/runtime/issues/51418#issuecomment-821729569. It would be quite surprising to me if his change were the cause of this, and I’d be sad to see it reverted.
Just an idea: Conservative GC stack scanning can create a phantom performance issues. Is there a way to tell whether the performance regression is due to GC taking more time?