runtime: ValueStringBuilder is slower at appending short strings than StringBuilder

Investigating why my PR dotnet/corefx#27250 makes HtmlEncode slower, I wrote a quick benchmark (using BenchmarkDotNet) to compare StringBuilder and ValueStringBuilder:

[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)]
[CategoriesColumn]
public class StringBuilderBench
{
    [Benchmark(Baseline = true), BenchmarkCategory("AppendChar")]
    public void StringBuilder_AppendChar()
    {
        var builder = new StringBuilder();
        for (var i = 0; i < 100000; i++)
        {
            builder.Append('<');
        }
        builder.ToString();
        builder.Clear();
    }
    [Benchmark, BenchmarkCategory("AppendChar")]
    public void ValueStringBuilder_AppendChar()
    {
        var builder = new ValueStringBuilder();
        for (var i = 0; i < 100000; i++)
        {
            builder.Append('<');
        }
        builder.ToString();
        builder.Dispose();
    }

    [Benchmark(Baseline = true), BenchmarkCategory("AppendString")]
    public void StringBuilder_AppendString()
    {
        var builder = new StringBuilder();
        for (var i = 0; i < 100000; i++)
        {
            builder.Append("&lt;");
        }
        builder.ToString();
        builder.Clear();
    }
    [Benchmark, BenchmarkCategory("AppendString")]
    public void ValueStringBuilder_AppendString()
    {
        var builder = new ValueStringBuilder();
        for (var i = 0; i < 100000; i++)
        {
            builder.Append("&lt;");
        }
        builder.ToString();
        builder.Dispose();
    }
}

Here are the results:

                          Method |   Categories |       Mean |     Error |    StdDev | Scaled | ScaledSD |
-------------------------------- |------------- |-----------:|----------:|----------:|-------:|---------:|
        StringBuilder_AppendChar |   AppendChar |   430.9 us |  4.072 us |  3.610 us |   1.00 |     0.00 |
   ValueStringBuilder_AppendChar |   AppendChar |   359.0 us |  5.030 us |  4.459 us |   0.83 |     0.01 |
                                 |              |            |           |           |        |          |
      StringBuilder_AppendString | AppendString | 1,518.1 us | 17.672 us | 16.530 us |   1.00 |     0.00 |
 ValueStringBuilder_AppendString | AppendString | 2,230.0 us | 31.823 us | 29.767 us |   1.47 |     0.02 |

(This is using .NET Core 2.1 RC1 on OSX.) So ValueStringBuilder is faster at appending single characters but slower at appending strings than StringBuilder.

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 33 (32 by maintainers)

Commits related to this issue

Most upvoted comments

ValueStringBuilder is slower when adding strings as it isn’t optimized for that

ValueStringBuilder should be as good with appending strings as it is with appending spans. You are likely just seeing a difference in fixed overhead for small strings. https://github.com/dotnet/coreclr/issues/18005 may be contributing to it as well. Make sure to pick it up when testing this.

@gfoidl please the next time post diff links instead of the direct commit as it helps us to easily spot the differences. https://help.github.com/articles/comparing-commits-across-time/

plus

  • avoid bound checks
  • reuse values from registers instead from the stack (Length)

Without the hint to inline it is faster than the current implementation, but still slower compared to StringBuilder.

I wonder what the numbers would be like if they were both presized.

There does not seem to be anything actionable here. dotnet/corefx#27250 that this issue is a follow up on was merged with nice improvements.

Can you also add benchmarks with longer strings?

Method Len Mean Error StdDev Scaled ScaledSD Gen 0 Gen 1 Gen 2 Allocated
StringBuilder_AppendLongString 10 1,337.6 us 26.217 us 23.241 us 1.00 0.00 332.0313 332.0313 332.0313 1565.16 KB
ValueStringBuilder0_AppendLongString 10 2,405.5 us 17.792 us 14.857 us 1.80 0.03 195.3125 195.3125 195.3125 781.28 KB
ValueStringBuilder2_AppendLongString 10 853.7 us 16.327 us 20.052 us 0.64 0.02 198.2422 198.2422 198.2422 781.28 KB
StringBuilder_AppendLongString 100 1,358.5 us 23.61 us 22.09 us 1.00 0.00 365.2344 359.3750 359.3750 1564.19 KB
ValueStringBuilder0_AppendLongString 100 2,418.8 us 35.65 us 33.35 us 1.78 0.04 195.3125 195.3125 195.3125 781.28 KB
ValueStringBuilder2_AppendLongString 100 837.0 us 16.55 us 21.51 us 0.62 0.02 198.2422 198.2422 198.2422 781.28 KB
StringBuilder_AppendLongString 1000 1,338.9 us 26.261 us 29.189 us 1.00 0.00 332.0313 332.0313 332.0313 1565.16 KB
ValueStringBuilder0_AppendLongString 1000 2,421.5 us 23.603 us 20.923 us 1.81 0.04 195.3125 195.3125 195.3125 781.28 KB
ValueStringBuilder2_AppendLongString 1000 858.6 us 17.063 us 38.515 us 0.64 0.03 198.2422 198.2422 198.2422 781.28 KB

I think we would rather want to fix the JIT bug than write unnatural code like this.

👍 Until the JIT is fixed we could use the workaround, then switch back to proper code.


please the next time post diff links instead of the direct commit as it helps us to easily spot the differences

Normally I would do a PR, so the diff is right there. Here I don’t know if it is good to go and the code lives in no fork, the implementations are side-by-side in files. Couldn’t figure out how to compare them in github. But I could post a (file-) diff… Anyway thanks for the link – didn’t know about the /compare in the url. Next time I’ll make it better.

reuse values from registers instead from the stack (Length)

This looks like a workaround for JIT bug. I think we would rather want to fix the JIT bug than write unnatural code like this.

Without the hint to inline it is faster than the current implementation, but still slower compared to StringBuilder.

StringBuilder does not use AggressiveInlining and it uses much more complex internal data structure than ValueStringBuilder, but it still faster? It sounds suspect to me. Also, AggressiveInlining works great for microbenchmarks, but it hurts at scale because of it introduces code bloat that makes things slower.