BenchmarkDotNet: Can't use latest version of BenchmarkDotNet
I tried to use the latest bdn master (dotnet/BenchmarkDotNet@73f8fd1dc49f34b79afa4ef16e0204b1369fdd1a) here, but trying to run benchmarks fails with:
Unhandled exception. System.InvalidOperationException: Failed to compare two elements in the array.
---> System.ArgumentException: At least one object must implement IComparable.
at System.Collections.Comparer.Compare(Object a, Object b)
at System.ValueTuple`3.CompareTo(ValueTuple`3 other)
at System.ValueTuple`3.System.IComparable.CompareTo(Object other)
at BenchmarkDotNet.Parameters.ParameterComparer.CompareValues(Object x, Object y)
at BenchmarkDotNet.Parameters.ParameterComparer.Compare(ParameterInstances x, ParameterInstances y)
at BenchmarkDotNet.Order.DefaultOrderer.BenchmarkComparer.Compare(BenchmarkCase x, BenchmarkCase y)
at System.Collections.Generic.ArraySortHelper`1.SwapIfGreater(Span`1 keys, Comparison`1 comparer, Int32 i, Int32 j)
at System.Collections.Generic.ArraySortHelper`1.PickPivotAndPartition(Span`1 keys, Comparison`1 comparer)
at System.Collections.Generic.ArraySortHelper`1.IntroSort(Span`1 keys, Int32 depthLimit, Comparison`1 comparer)
at System.Collections.Generic.ArraySortHelper`1.IntrospectiveSort(Span`1 keys, Comparison`1 comparer)
at System.Collections.Generic.GenericArraySortHelper`1.Sort(Span`1 keys, IComparer`1 comparer)
--- End of inner exception stack trace ---
at System.Collections.Generic.GenericArraySortHelper`1.Sort(Span`1 keys, IComparer`1 comparer)
at System.Array.Sort[T](T[] array, Int32 index, Int32 length, IComparer`1 comparer)
at System.Collections.Generic.List`1.Sort(Int32 index, Int32 count, IComparer`1 comparer)
at BenchmarkDotNet.Order.DefaultOrderer.GetExecutionOrder(ImmutableArray`1 benchmarkCases, IEnumerable`1 order)
at BenchmarkDotNet.Running.BenchmarkConverter.MethodsToBenchmarksWithFullConfig(Type type, MethodInfo[] benchmarkMethods, IConfig config)
at BenchmarkDotNet.Running.BenchmarkConverter.TypeToBenchmarks(Type type, IConfig config)
at BenchmarkDotNet.Running.TypeFilter.<>c__DisplayClass1_0.<Filter>b__0(Type type)
at System.Linq.Enumerable.SelectListIterator`2.MoveNext()
at System.Linq.Enumerable.WhereEnumerableIterator`1.ToArray()
at BenchmarkDotNet.Running.TypeFilter.Filter(IConfig effectiveConfig, IEnumerable`1 types)
at BenchmarkDotNet.Running.BenchmarkSwitcher.RunWithDirtyAssemblyResolveHelper(String[] args, IConfig config, Boolean askUserForInput)
at BenchmarkDotNet.Running.BenchmarkSwitcher.Run(String[] args, IConfig config)
at MicroBenchmarks.Program.Main(String[] args) in /Users/ankj/dev/performance/src/benchmarks/micro/Program.cs:line 43
This seems to be failing for
System.ValueTuple`3[System.Globalization.CultureInfo,System.Globalization.CompareOptions,System.Boolean], (en-US, Ordinal, False)
System.ValueTuple`3[System.Globalization.CultureInfo,System.Globalization.CompareOptions,System.Boolean], (pl-PL, None, False)`
… , because CultureInfo does not implement IComparable.
IIUC, this is a new code path introduced as part of https://github.com/dotnet/BenchmarkDotNet/pull/2304 .
And it fails for https://github.com/dotnet/performance/blob/dd14c7d44444a0211035af8464b743a55b4dd55e/src/benchmarks/micro/libraries/System.Globalization/StringSearch.cs#L46 .
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 18 (13 by maintainers)
I also find that acceptable. Just catch the exact exception type and compare the message to avoid suppressing erroneous exceptions.
System.ArgumentException: At least one object must implement IComparable.Going through
ITuplemight make that easier.