runtime: Perf regression in string handling / culture ICU

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline bbf4e797a13984acee37b19d0a79217188824301
Compare a53ec50f18af59d864d582d561165ac21ea25089
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToLowerInvariant - Duration of single invocation 47.57 ns 52.14 ns 1.10 0.01 False
ToUpperInvariant - Duration of single invocation 14.22 ns 18.41 ns 1.29 0.11 False
ToUpperInvariant - Duration of single invocation 14.34 ns 19.17 ns 1.34 0.11 False
IndexerCheckPathLength - Duration of single invocation 146.05 ns 165.12 ns 1.13 0.07 False
Replace_Char - Duration of single invocation 4.37 ns 7.14 ns 1.63 0.17 False
Remove_IntInt - Duration of single invocation 10.73 ns 12.10 ns 1.13 0.16 False

graph graph graph graph graph graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline Compare

Histogram

System.Tests.Perf_String.ToLowerInvariant(s: “This is a much longer piece of text that might benefit more from vectorization.”)

System.Tests.Perf_String.ToUpperInvariant(s: “TeSt”)

System.Tests.Perf_String.ToUpperInvariant(s: “test”)

System.Tests.Perf_String.IndexerCheckPathLength

System.Tests.Perf_String.Replace_Char(text: “Hello”, oldChar: ‘a’, newChar: ‘b’)

System.Tests.Perf_String.Remove_IntInt(s: “dzsdzsDDZSDZSDZSddsz”, i1: 0, i2: 8)

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline bbf4e797a13984acee37b19d0a79217188824301
Compare a53ec50f18af59d864d582d561165ac21ea25089
Diff Diff

Regressions in System.Formats.Cbor.Tests.Perf_CborReader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SkipValue - Duration of single invocation 60.73 ns 70.48 ns 1.16 0.11 False
SkipValue - Duration of single invocation 43.26 ns 49.98 ns 1.16 0.07 False

graph graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Formats.Cbor.Tests.Perf_CborReader*'

Payloads

Baseline Compare

Histogram

System.Formats.Cbor.Tests.Perf_CborReader.SkipValue(encoding: (Text String, Strict))

System.Formats.Cbor.Tests.Perf_CborReader.SkipValue(encoding: (Byte String, Strict))

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline bbf4e797a13984acee37b19d0a79217188824301
Compare a53ec50f18af59d864d582d561165ac21ea25089
Diff Diff

Regressions in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfString - Duration of single invocation 1.41 μs 1.52 μs 1.08 0.01 False
IndexOfString - Duration of single invocation 29.80 ns 35.54 ns 1.19 0.10 False
Trim - Duration of single invocation 7.18 ns 8.68 ns 1.21 0.02 False
IndexOfString - Duration of single invocation 26.32 ns 33.14 ns 1.26 0.03 False
IndexOfString - Duration of single invocation 18.71 ns 21.89 ns 1.17 0.08 False
IndexOfString - Duration of single invocation 26.23 ns 33.04 ns 1.26 0.03 False

graph graph graph graph graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline Compare

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: “だ”, value: “た”, comparisonType: InvariantCulture)

System.Memory.ReadOnlySpan.IndexOfString(input: “More Test’s”, value: “Tests”, comparisonType: OrdinalIgnoreCase)

System.Memory.ReadOnlySpan.Trim(input: " abcdefg ")

System.Memory.ReadOnlySpan.IndexOfString(input: “だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだxだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ”, value: “x”, comparisonType: Ordinal)

System.Memory.ReadOnlySpan.IndexOfString(input: “StrIng”, value: “string”, comparisonType: OrdinalIgnoreCase)

System.Memory.ReadOnlySpan.IndexOfString(input: “AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA”, value: “X”, comparisonType: Ordinal)

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline bbf4e797a13984acee37b19d0a79217188824301
Compare a53ec50f18af59d864d582d561165ac21ea25089
Diff Diff

Regressions in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ICollection - Duration of single invocation 825.39 μs 899.29 μs 1.09 0.06 False

graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'

Payloads

Baseline Compare

Histogram

System.Collections.ContainsFalse<String>.ICollection(Size: 512)

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline bbf4e797a13984acee37b19d0a79217188824301
Compare a53ec50f18af59d864d582d561165ac21ea25089
Diff Diff

Regressions in System.Tests.Perf_HashCode

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Add - Duration of single invocation 2.46 μs 3.07 μs 1.25 0.11 False

graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_HashCode*'

Payloads

Baseline Compare

Histogram

System.Tests.Perf_HashCode.Add

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline bbf4e797a13984acee37b19d0a79217188824301
Compare a53ec50f18af59d864d582d561165ac21ea25089
Diff Diff

Regressions in System.Collections.IterateForEachNonGeneric<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ArrayList - Duration of single invocation 3.02 μs 3.61 μs 1.19 0.17 False

graph Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEachNonGeneric&lt;Int32&gt;*'

Payloads

Baseline Compare

Histogram

System.Collections.IterateForEachNonGeneric<Int32>.ArrayList(Size: 512)

Docs

Profiling workflow for dotnet/runtime repository Benchmarking workflow for dotnet/runtime repository

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 24 (24 by maintainers)

Most upvoted comments

Looking at your [first URL ](https://pvscmdupload.blob.core.windows.net/reports/allTestHistory%2Frefs%2Fheads%2Fmain_x64_Windows 10.0.19042%2Famd%2FSystem.Memory.ReadOnlySpan.IndexOfString(input%3A "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ.html) Kunal – did it fix itself? I’m not sure if I’m looking at the right graph: image

+1
danmoseley on Aug 31, 2021
Read more comments on GitHub
← runtime: Perf regression for Math.Min and Math.Max
runtime: [Perf] Regression in System.Collections.CtorFromCollection<String>.ConcurrentQueue →