runtime: Perf: double.ToString(CultureInfo) 7x slower on Linux
@ianhays commented on Tue Oct 13 2015
| test name | date ran | Windows time | Linux time | linux/windows |
|---|---|---|---|---|
| System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: “fr”) | 10/13/2015 | 3.780472576 | 29.0264 | 7.677981896 |
| System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: “da”) | 10/13/2015 | 3.757379192 | 28.05800001 | 7.467439025 |
| System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: “ja”) | 10/13/2015 | 3.773344988 | 27.9326 | 7.402609643 |
| System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: “”) | 10/13/2015 | 3.806987203 | 27.96 | 7.34439033 |
Perf Test:
[Benchmark]
[InlineData("fr")]
[InlineData("da")]
[InlineData("ja")]
[InlineData("")]
public void ToString(string culturestring)
{
double number = 104234.343;
CultureInfo cultureInfo = new CultureInfo(culturestring);
foreach (var iteration in Benchmark.Iterations)
using (iteration.StartMeasurement())
{
for (int i = 0; i < innerIterations; i++)
{
number.ToString(cultureInfo); number.ToString(cultureInfo); number.ToString(cultureInfo);
number.ToString(cultureInfo); number.ToString(cultureInfo); number.ToString(cultureInfo);
number.ToString(cultureInfo); number.ToString(cultureInfo); number.ToString(cultureInfo);
}
}
}
Although it’s an admittedly poor comparative data point, the ToString(CultureInfo) method for DateTime was 1:1 between Linux:Windows so not all ToString methods are equally slow when using CultureInfo. As such, I am unsure of whom to to notify; @steveharter @stephentoub?
I have not yet added equivalent ToString tests to other classes, but can if desired for more data points.
@steveharter commented on Mon Nov 16 2015
@ianhays is 7x on windows with coreclr, or desktop clr? In a similar test, I’m getting: coreclr+windows 4.3x faster than coreclr+linux desktopclr+windows 18x-25x faster than coreclr+linux This includes using a simple cpu-only baseline looping test used to normalize the results due to different cpu speeds and\or differences in clr jitting\etc.
@ianhays commented on Mon Dec 26 2016
It’s interesting that you’re seeing such a big difference between Windows-CoreCLR and Windows-Desktop. For my tests they were nearly 1:1. Could you post some numerical results and we can compare data?
I’m running on Ubuntu 14.04 (running in HyperV), CentOS 7.1 (running in HyperV), and Windows 10 running on coreclr. Same computer. Exact test listed above. Using binaries from \cpvsbuild\Drops\dev14\ProjectK\raw\23513.00\binaries.amd64ret.
CentOS results:
<collection total="4" passed="4" failed="0" skipped="0" name="Test collection for System.Globalization.Tests.Perf_NumberCultureInfo" time="4.209">
<test name="System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: "fr")" type="System.Globalization.Tests.Perf_NumberCultureInfo" method="ToString" time="1.0664355" result="Pass">
<traits>
<trait name="Benchmark" value="true" />
</traits>
<performance runid="System.Globalization.Tests.dll-CentOS" etl="/home/ianha/rel/results/System.Globalization.Tests.dll-CentOS.csv">
<metrics>
<Duration displayName="Duration" unit="msec" />
</metrics>
<iterations>
<iteration index="0" Duration="29.394985914230347" />
<iteration index="1" Duration="27.538418054580688" />
<iteration index="2" Duration="28.897694945335388" />
<iteration index="3" Duration="33.568613886833191" />
<iteration index="4" Duration="28.437203049659729" />
<iteration index="5" Duration="27.611016988754272" />
<iteration index="6" Duration="29.217790007591248" />
<iteration index="7" Duration="28.009710907936096" />
<iteration index="8" Duration="28.690199017524719" />
<iteration index="9" Duration="31.641247987747192" />
<iteration index="10" Duration="29.398486018180847" />
<iteration index="11" Duration="27.699916005134583" />
<iteration index="12" Duration="27.58421802520752" />
<iteration index="13" Duration="27.673416018486023" />
<iteration index="14" Duration="28.14900803565979" />
<iteration index="15" Duration="27.921112060546875" />
<iteration index="16" Duration="27.766514897346497" />
<iteration index="17" Duration="27.707115888595581" />
<iteration index="18" Duration="28.539501070976257" />
<iteration index="19" Duration="27.511018991470337" />
<iteration index="20" Duration="27.805315017700195" />
<iteration index="21" Duration="27.854113936424255" />
<iteration index="22" Duration="27.897012948989868" />
<iteration index="23" Duration="31.014458060264587" />
<iteration index="24" Duration="28.17520797252655" />
<iteration index="25" Duration="28.119809031486511" />
<iteration index="26" Duration="28.370004057884216" />
<iteration index="27" Duration="27.953910946846008" />
<iteration index="28" Duration="28.936293959617615" />
<iteration index="29" Duration="29.012992978096008" />
<iteration index="30" Duration="28.124009013175964" />
<iteration index="31" Duration="27.519919991493225" />
<iteration index="32" Duration="27.68511700630188" />
<iteration index="33" Duration="27.905812978744507" />
<iteration index="34" Duration="27.937612056732178" />
<iteration index="35" Duration="27.705316066741943" />
</iterations>
</performance>
</test>
<test name="System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: "da")" type="System.Globalization.Tests.Perf_NumberCultureInfo" method="ToString" time="1.0456405" result="Pass">
<traits>
<trait name="Benchmark" value="true" />
</traits>
<performance runid="System.Globalization.Tests.dll-CentOS" etl="/home/ianha/rel/results/System.Globalization.Tests.dll-CentOS.csv">
<metrics>
<Duration displayName="Duration" unit="msec" />
</metrics>
<iterations>
<iteration index="0" Duration="36.094968914985657" />
<iteration index="1" Duration="27.743814945220947" />
<iteration index="2" Duration="27.875913023948669" />
<iteration index="3" Duration="27.602617979049683" />
<iteration index="4" Duration="27.816213965415955" />
<iteration index="5" Duration="28.792296886444092" />
<iteration index="6" Duration="27.533019065856934" />
<iteration index="7" Duration="28.085710048675537" />
<iteration index="8" Duration="28.151907920837402" />
<iteration index="9" Duration="27.482020020484924" />
<iteration index="10" Duration="27.996210932731628" />
<iteration index="11" Duration="28.618600010871887" />
<iteration index="12" Duration="27.751914978027344" />
<iteration index="13" Duration="27.638016939163208" />
<iteration index="14" Duration="27.872413039207458" />
<iteration index="15" Duration="30.500468015670776" />
<iteration index="16" Duration="27.872312903404236" />
<iteration index="17" Duration="27.939612030982971" />
<iteration index="18" Duration="27.825913071632385" />
<iteration index="19" Duration="27.495819926261902" />
<iteration index="20" Duration="27.704716086387634" />
<iteration index="21" Duration="27.9049129486084" />
<iteration index="22" Duration="27.931511998176575" />
<iteration index="23" Duration="27.667815923690796" />
<iteration index="24" Duration="27.3945209980011" />
<iteration index="25" Duration="27.917912006378174" />
<iteration index="26" Duration="27.716416001319885" />
<iteration index="27" Duration="27.933212041854858" />
<iteration index="28" Duration="27.486520051956177" />
<iteration index="29" Duration="29.66008198261261" />
<iteration index="30" Duration="29.410285949707031" />
<iteration index="31" Duration="28.070809960365295" />
<iteration index="32" Duration="27.493919968605042" />
<iteration index="33" Duration="28.041810035705566" />
<iteration index="34" Duration="28.174207925796509" />
<iteration index="35" Duration="27.583817958831787" />
<iteration index="36" Duration="27.807613968849182" />
</iterations>
</performance>
</test>
<test name="System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: "ja")" type="System.Globalization.Tests.Perf_NumberCultureInfo" method="ToString" time="1.0497453" result="Pass">
<traits>
<trait name="Benchmark" value="true" />
</traits>
<performance runid="System.Globalization.Tests.dll-CentOS" etl="/home/ianha/rel/results/System.Globalization.Tests.dll-CentOS.csv">
<metrics>
<Duration displayName="Duration" unit="msec" />
</metrics>
<iterations>
<iteration index="0" Duration="28.615100026130676" />
<iteration index="1" Duration="27.676715970039368" />
<iteration index="2" Duration="27.601817011833191" />
<iteration index="3" Duration="27.763615012168884" />
<iteration index="4" Duration="28.683698058128357" />
<iteration index="5" Duration="27.865113019943237" />
<iteration index="6" Duration="27.877612948417664" />
<iteration index="7" Duration="27.683015942573547" />
<iteration index="8" Duration="28.065410017967224" />
<iteration index="9" Duration="27.664615988731384" />
<iteration index="10" Duration="36.985154032707214" />
<iteration index="11" Duration="30.639065027236938" />
<iteration index="12" Duration="27.659315943717957" />
<iteration index="13" Duration="27.68911600112915" />
<iteration index="14" Duration="27.640017032623291" />
<iteration index="15" Duration="27.554818987846375" />
<iteration index="16" Duration="27.4220210313797" />
<iteration index="17" Duration="27.686416029930115" />
<iteration index="18" Duration="27.96251106262207" />
<iteration index="19" Duration="27.818414092063904" />
<iteration index="20" Duration="28.181107044219971" />
<iteration index="21" Duration="30.207971930503845" />
<iteration index="22" Duration="27.66921591758728" />
<iteration index="23" Duration="27.575117945671082" />
<iteration index="24" Duration="27.539119005203247" />
<iteration index="25" Duration="27.812513947486877" />
<iteration index="26" Duration="28.019811034202576" />
<iteration index="27" Duration="27.516318917274475" />
<iteration index="28" Duration="27.816313982009888" />
<iteration index="29" Duration="27.987210988998413" />
<iteration index="30" Duration="28.7555969953537" />
<iteration index="31" Duration="27.774914979934692" />
<iteration index="32" Duration="28.356604933738708" />
<iteration index="33" Duration="28.666599035263062" />
<iteration index="34" Duration="27.587418079376221" />
<iteration index="35" Duration="30.257572054862976" />
<iteration index="36" Duration="28.29650604724884" />
</iterations>
</performance>
</test>
<test name="System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: "")" type="System.Globalization.Tests.Perf_NumberCultureInfo" method="ToString" time="1.0471872" result="Pass">
<traits>
<trait name="Benchmark" value="true" />
</traits>
<performance runid="System.Globalization.Tests.dll-CentOS" etl="/home/ianha/rel/results/System.Globalization.Tests.dll-CentOS.csv">
<metrics>
<Duration displayName="Duration" unit="msec" />
</metrics>
<iterations>
<iteration index="0" Duration="28.685899019241333" />
<iteration index="1" Duration="27.759415030479431" />
<iteration index="2" Duration="28.962594032287598" />
<iteration index="3" Duration="27.831114053726196" />
<iteration index="4" Duration="27.918012022972107" />
<iteration index="5" Duration="27.727115988731384" />
<iteration index="6" Duration="27.582617998123169" />
<iteration index="7" Duration="28.238106966018677" />
<iteration index="8" Duration="27.707015991210938" />
<iteration index="9" Duration="27.508718967437744" />
<iteration index="10" Duration="27.685816049575806" />
<iteration index="11" Duration="27.990810990333557" />
<iteration index="12" Duration="36.150267958641052" />
<iteration index="13" Duration="27.81671404838562" />
<iteration index="14" Duration="27.862313032150269" />
<iteration index="15" Duration="28.303506016731262" />
<iteration index="16" Duration="27.79661500453949" />
<iteration index="17" Duration="28.096608996391296" />
<iteration index="18" Duration="27.796614050865173" />
<iteration index="19" Duration="28.159507989883423" />
<iteration index="20" Duration="28.012010931968689" />
<iteration index="21" Duration="27.406520962715149" />
<iteration index="22" Duration="28.078809022903442" />
<iteration index="23" Duration="27.770314931869507" />
<iteration index="24" Duration="27.843314051628113" />
<iteration index="25" Duration="27.394421100616455" />
<iteration index="26" Duration="29.00989294052124" />
<iteration index="27" Duration="30.44566810131073" />
<iteration index="28" Duration="28.251506090164185" />
<iteration index="29" Duration="28.132707953453064" />
<iteration index="30" Duration="28.735197067260742" />
<iteration index="31" Duration="28.501802086830139" />
<iteration index="32" Duration="27.959811925888062" />
<iteration index="33" Duration="27.853813886642456" />
<iteration index="34" Duration="27.863412976264954" />
<iteration index="35" Duration="27.652817010879517" />
<iteration index="36" Duration="27.678416967391968" />
</iterations>
</performance>
</test>
</collection>
Windows 10 on CoreCLR results:
<collection total="4" passed="4" failed="0" skipped="0" name="Test collection for System.Globalization.Tests.Perf_NumberCultureInfo" time="4.087">
<test name="System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: "fr")" type="System.Globalization.Tests.Perf_NumberCultureInfo" method="ToString" time="1.075388" result="Pass">
<traits>
<trait name="Benchmark" value="true" />
</traits>
<performance runid="System.Globalization.Tests.dll-WindowsCore" etl="D:\tools\testing\relresults\11-16-2015\System.Globalization.Tests.dll-WindowsCore.etl">
<metrics>
<Duration displayName="Duration" unit="msec" />
</metrics>
<iterations>
<iteration index="0" Duration="2.1205962986784925" />
<iteration index="1" Duration="1.544973291548672" />
<iteration index="2" Duration="1.4910888198560315" />
<iteration index="3" Duration="1.5059141771470195" />
<iteration index="4" Duration="1.6136831205324143" />
<iteration index="5" Duration="1.5338542735803458" />
<iteration index="6" Duration="1.5224501525871119" />
<iteration index="7" Duration="1.496505777327684" />
<iteration index="8" Duration="1.5469690127224567" />
<iteration index="9" Duration="1.6091214721351434" />
<iteration index="10" Duration="1.6191000780041804" />
<iteration index="11" Duration="1.5027780438739455" />
<iteration index="12" Duration="1.9124710905532538" />
<iteration index="13" Duration="1.4910888198559178" />
<iteration index="14" Duration="1.4896633047318346" />
<iteration index="15" Duration="1.5284373161084659" />
<iteration index="16" Duration="1.5355648917292228" />
<iteration index="17" Duration="1.582606890826014" />
<iteration index="18" Duration="1.4899484077566285" />
<iteration index="19" Duration="1.4959355712779825" />
<iteration index="20" Duration="1.5515306611197275" />
<iteration index="21" Duration="1.5823217878012201" />
<iteration index="22" Duration="1.4919441289304132" />
<iteration index="23" Duration="1.6273680657242267" />
<iteration index="24" Duration="1.496505777327684" />
<iteration index="25" Duration="1.6883801130376241" />
<iteration index="26" Duration="1.4970759833772718" />
<iteration index="27" Duration="1.4885228926324316" />
<iteration index="28" Duration="1.5184587102395426" />
<iteration index="29" Duration="1.6456146593133099" />
<iteration index="30" Duration="1.5187438132643365" />
<iteration index="31" Duration="1.5190289162891304" />
<iteration index="32" Duration="1.4899484077566285" />
<iteration index="33" Duration="1.5090503104200934" />
<iteration index="34" Duration="1.4967908803524779" />
<iteration index="35" Duration="1.5107609285691979" />
<iteration index="36" Duration="1.5928705997198449" />
<iteration index="37" Duration="1.5743389031059678" />
<iteration index="38" Duration="1.6444742472139069" />
<iteration index="39" Duration="1.5680666365597062" />
<iteration index="40" Duration="1.611402296333722" />
<iteration index="41" Duration="1.5652156063114262" />
<iteration index="42" Duration="1.5800409636026416" />
<iteration index="43" Duration="1.5036333529484409" />
<iteration index="44" Duration="1.49336964405461" />
<iteration index="45" Duration="1.4950802622036008" />
<iteration index="46" Duration="1.5247309767856905" />
<iteration index="47" Duration="1.489378201706927" />
<iteration index="48" Duration="1.5746240061308754" />
<iteration index="49" Duration="1.49622067430289" />
<iteration index="50" Duration="1.4967908803525916" />
<iteration index="51" Duration="1.5811813757019308" />
<iteration index="52" Duration="1.5501051459955306" />
<iteration index="53" Duration="1.5047737650477302" />
<iteration index="54" Duration="1.5401265401266073" />
<iteration index="55" Duration="1.5412669522258966" />
<iteration index="56" Duration="1.4902335107814224" />
<iteration index="57" Duration="1.4890930986822468" />
<iteration index="58" Duration="1.4839612442351608" />
<iteration index="59" Duration="1.5672113274853245" />
<iteration index="60" Duration="1.4896633047318346" />
<iteration index="61" Duration="1.6245170354759466" />
<iteration index="62" Duration="1.5552370004425029" />
<iteration index="63" Duration="1.5164629890657579" />
<iteration index="64" Duration="1.6176745628799836" />
<iteration index="65" Duration="1.5663560184108292" />
<iteration index="66" Duration="1.513326855792684" />
<iteration index="67" Duration="1.6843886706900548" />
<iteration index="68" Duration="1.8161062681610929" />
<iteration index="69" Duration="1.5187438132643365" />
<iteration index="70" Duration="1.5660709153859216" />
<iteration index="71" Duration="1.4927994380049086" />
<iteration index="72" Duration="1.5392712310519983" />
<iteration index="73" Duration="1.5635049881625491" />
<iteration index="74" Duration="1.5463988066728689" />
<iteration index="75" Duration="1.5161778860408504" />
<iteration index="76" Duration="1.4930845410297025" />
<iteration index="77" Duration="1.5840324059502109" />
<iteration index="78" Duration="1.4899484077566285" />
<iteration index="79" Duration="1.5153225769664687" />
<iteration index="80" Duration="1.6145384296069096" />
<iteration index="81" Duration="1.5769048303295676" />
<iteration index="82" Duration="1.5161778860408504" />
<iteration index="83" Duration="1.5119013406684871" />
<iteration index="84" Duration="1.5592284427900722" />
<iteration index="85" Duration="1.4879526865827302" />
<iteration index="86" Duration="1.5908748785460602" />
<iteration index="87" Duration="1.5081950013457117" />
<iteration index="88" Duration="1.6784015071685872" />
<iteration index="89" Duration="1.8466122918176779" />
<iteration index="90" Duration="1.7445454089289569" />
<iteration index="91" Duration="1.564930503286746" />
<iteration index="92" Duration="1.7029203673039319" />
<iteration index="93" Duration="1.5272969040092903" />
<iteration index="94" Duration="1.5264415949347949" />
<iteration index="95" Duration="1.5663560184108292" />
<iteration index="96" Duration="1.5592284427900722" />
<iteration index="97" Duration="1.525301182835392" />
<iteration index="98" Duration="1.8417655403957269" />
<iteration index="99" Duration="1.5506753520451184" />
<iteration index="100" Duration="1.523305461661721" />
</iterations>
</performance>
</test>
<test name="System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: "da")" type="System.Globalization.Tests.Perf_NumberCultureInfo" method="ToString" time="1.0043532" result="Pass">
<traits>
<trait name="Benchmark" value="true" />
</traits>
<performance runid="System.Globalization.Tests.dll-WindowsCore" etl="D:\tools\testing\relresults\11-16-2015\System.Globalization.Tests.dll-WindowsCore.etl">
<metrics>
<Duration displayName="Duration" unit="msec" />
</metrics>
<iterations>
<iteration index="0" Duration="1.8386294071224256" />
<iteration index="1" Duration="1.6105469872593403" />
<iteration index="2" Duration="1.6895205251369134" />
<iteration index="3" Duration="1.5780452424287432" />
<iteration index="4" Duration="1.4999270136256655" />
<iteration index="5" Duration="1.4959355712778688" />
<iteration index="6" Duration="1.4985014985013549" />
<iteration index="7" Duration="1.708622427800492" />
<iteration index="8" Duration="1.5258713888852071" />
<iteration index="9" Duration="1.4979312924517671" />
<iteration index="10" Duration="1.5375606129030075" />
<iteration index="11" Duration="1.5429775703748874" />
<iteration index="12" Duration="1.5999981753407155" />
<iteration index="13" Duration="1.7941533352493479" />
<iteration index="14" Duration="1.4959355712780962" />
<iteration index="15" Duration="1.6005683813903033" />
<iteration index="16" Duration="1.4950802622036008" />
<iteration index="17" Duration="1.6176745628799836" />
<iteration index="18" Duration="1.6279382717739281" />
<iteration index="19" Duration="1.5831770968757155" />
<iteration index="20" Duration="1.5150374739416748" />
<iteration index="21" Duration="1.4927994380050222" />
<iteration index="22" Duration="1.7225924760170983" />
<iteration index="23" Duration="1.5463988066728689" />
<iteration index="24" Duration="1.4922292319552071" />
<iteration index="25" Duration="1.5418371582754844" />
<iteration index="26" Duration="1.6863843918638395" />
<iteration index="27" Duration="1.6866694948887471" />
<iteration index="28" Duration="1.5093354134451147" />
<iteration index="29" Duration="1.6019938965143865" />
<iteration index="30" Duration="1.5332840675305306" />
<iteration index="31" Duration="1.6079810600358542" />
<iteration index="32" Duration="1.8936542909145828" />
<iteration index="33" Duration="1.5150374739414474" />
<iteration index="34" Duration="1.7682089599898063" />
<iteration index="35" Duration="1.8825352729461429" />
<iteration index="36" Duration="1.6347807443696638" />
<iteration index="37" Duration="1.5888791573722756" />
<iteration index="38" Duration="1.6230915203518634" />
<iteration index="39" Duration="1.6667122831506731" />
<iteration index="40" Duration="1.4987866015262625" />
<iteration index="41" Duration="1.6812525374170946" />
<iteration index="42" Duration="1.5592284427900722" />
<iteration index="43" Duration="1.5680666365597062" />
<iteration index="44" Duration="1.8534547644137547" />
<iteration index="45" Duration="1.7160351064460428" />
<iteration index="46" Duration="1.7049160884776029" />
<iteration index="47" Duration="1.9942956586792207" />
<iteration index="48" Duration="1.5170331951155731" />
<iteration index="49" Duration="1.7921576140754496" />
<iteration index="50" Duration="1.608551266085442" />
<iteration index="51" Duration="1.5689219456342016" />
<iteration index="52" Duration="1.6219511082524605" />
<iteration index="53" Duration="1.6099767812097525" />
<iteration index="54" Duration="2.0721287844576182" />
<iteration index="55" Duration="1.8158211651361853" />
<iteration index="56" Duration="1.5566625155668135" />
<iteration index="57" Duration="1.5042035589981424" />
<iteration index="58" Duration="1.5406967461760814" />
<iteration index="59" Duration="1.7171755185452184" />
<iteration index="60" Duration="1.6071257509613588" />
<iteration index="61" Duration="1.6002832783653957" />
<iteration index="62" Duration="1.532143655431355" />
<iteration index="63" Duration="1.6516018228346638" />
<iteration index="64" Duration="2.1553788677076682" />
<iteration index="65" Duration="1.7268690213895752" />
<iteration index="66" Duration="1.6550230591326454" />
<iteration index="67" Duration="1.5190289162892441" />
<iteration index="68" Duration="1.5942961148441555" />
<iteration index="69" Duration="1.5837473029253033" />
<iteration index="70" Duration="1.8617227521335735" />
<iteration index="71" Duration="1.68780990698815" />
<iteration index="72" Duration="1.7074820157013164" />
<iteration index="73" Duration="1.8349230677997639" />
<iteration index="74" Duration="1.6838184646403533" />
<iteration index="75" Duration="1.6661420771010853" />
<iteration index="76" Duration="1.7034905733537471" />
<iteration index="77" Duration="1.6695633133988395" />
<iteration index="78" Duration="1.8697056368289395" />
<iteration index="79" Duration="1.6803972283423718" />
<iteration index="80" Duration="1.6977885128569596" />
<iteration index="81" Duration="1.6652867680263626" />
<iteration index="82" Duration="1.7111883550239781" />
<iteration index="83" Duration="1.9603683987245404" />
<iteration index="84" Duration="1.6732696527217286" />
<iteration index="85" Duration="1.6752653738956269" />
<iteration index="86" Duration="1.6635761498775992" />
<iteration index="87" Duration="1.6983587189067748" />
<iteration index="88" Duration="1.6709888285229226" />
<iteration index="89" Duration="1.6912311432859042" />
<iteration index="90" Duration="1.8292210073032038" />
<iteration index="91" Duration="1.6735547557464088" />
<iteration index="92" Duration="1.8876671273933425" />
<iteration index="93" Duration="1.6071257509613588" />
<iteration index="94" Duration="1.5734835940315861" />
<iteration index="95" Duration="1.7231626820669135" />
<iteration index="96" Duration="1.7476815422021446" />
<iteration index="97" Duration="1.5626496790880537" />
<iteration index="98" Duration="1.5706325637834198" />
<iteration index="99" Duration="1.5863132301487894" />
<iteration index="100" Duration="1.5894493634218634" />
</iterations>
</performance>
</test>
<test name="System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: "ja")" type="System.Globalization.Tests.Perf_NumberCultureInfo" method="ToString" time="1.0038747" result="Pass">
<traits>
<trait name="Benchmark" value="true" />
</traits>
<performance runid="System.Globalization.Tests.dll-WindowsCore" etl="D:\tools\testing\relresults\11-16-2015\System.Globalization.Tests.dll-WindowsCore.etl">
<metrics>
<Duration displayName="Duration" unit="msec" />
</metrics>
<iterations>
<iteration index="0" Duration="1.6245170354759466" />
<iteration index="1" Duration="1.5646454002617247" />
<iteration index="2" Duration="1.5215948435129576" />
<iteration index="3" Duration="1.5683517395846138" />
<iteration index="4" Duration="1.5281522130840131" />
<iteration index="5" Duration="1.5729133879822257" />
<iteration index="6" Duration="1.6721292406223256" />
<iteration index="7" Duration="1.6216660052277803" />
<iteration index="8" Duration="1.6872397009383349" />
<iteration index="9" Duration="1.6838184646403533" />
<iteration index="10" Duration="1.7280094334892055" />
<iteration index="11" Duration="1.6678526952500761" />
<iteration index="12" Duration="1.6344956413449836" />
<iteration index="13" Duration="1.7647877236918248" />
<iteration index="14" Duration="1.73171577281164" />
<iteration index="15" Duration="2.0265123004846828" />
<iteration index="16" Duration="1.7023501612543441" />
<iteration index="17" Duration="1.7990000866711853" />
<iteration index="18" Duration="1.8953649090635736" />
<iteration index="19" Duration="1.9843170528101837" />
<iteration index="20" Duration="1.7014948521800761" />
<iteration index="21" Duration="1.8568760007115088" />
<iteration index="22" Duration="1.8936542909145828" />
<iteration index="23" Duration="1.8309316254521946" />
<iteration index="24" Duration="1.7012097491547138" />
<iteration index="25" Duration="2.1180303714550064" />
<iteration index="26" Duration="1.84290595249513" />
<iteration index="27" Duration="1.9027775877093518" />
<iteration index="28" Duration="2.02622719746023" />
<iteration index="29" Duration="1.9769043741644055" />
<iteration index="30" Duration="1.7901618929013239" />
<iteration index="31" Duration="1.7539538087480651" />
<iteration index="32" Duration="1.9971466889278418" />
<iteration index="33" Duration="1.6054151328125954" />
<iteration index="34" Duration="1.6362062594939744" />
<iteration index="35" Duration="1.6812525374170946" />
<iteration index="36" Duration="1.7796130809829265" />
<iteration index="37" Duration="1.7556644268975106" />
<iteration index="38" Duration="1.6644314589520945" />
<iteration index="39" Duration="1.5555221034669557" />
<iteration index="40" Duration="1.727154124414028" />
<iteration index="41" Duration="1.5458286006232811" />
<iteration index="42" Duration="1.5714878728576878" />
<iteration index="43" Duration="1.8409102313212315" />
<iteration index="44" Duration="1.8332124496510005" />
<iteration index="45" Duration="1.7559495299224182" />
<iteration index="46" Duration="1.5398414371015861" />
<iteration index="47" Duration="1.6145384296069096" />
<iteration index="48" Duration="1.7094777368752148" />
<iteration index="49" Duration="1.6592996045051223" />
<iteration index="50" Duration="1.6815376404420022" />
<iteration index="51" Duration="1.5999981753407155" />
<iteration index="52" Duration="1.61026188423466" />
<iteration index="53" Duration="1.586598333173697" />
<iteration index="54" Duration="1.64875079258627" />
<iteration index="55" Duration="1.6481805865364549" />
<iteration index="56" Duration="1.5444030854987432" />
<iteration index="57" Duration="1.538986128027318" />
<iteration index="58" Duration="2.1610809282042283" />
<iteration index="59" Duration="1.7188861366944366" />
<iteration index="60" Duration="1.8232338437819635" />
<iteration index="61" Duration="1.653312440983882" />
<iteration index="62" Duration="1.5592284427898448" />
<iteration index="63" Duration="1.5529561762441517" />
<iteration index="64" Duration="1.562934782112734" />
<iteration index="65" Duration="1.5903046724965861" />
<iteration index="66" Duration="1.5358499947542441" />
<iteration index="67" Duration="1.6937970705093903" />
<iteration index="68" Duration="1.6476103804870945" />
<iteration index="69" Duration="1.6396274957919559" />
<iteration index="70" Duration="1.5024929408491516" />
<iteration index="71" Duration="1.6744100648211315" />
<iteration index="72" Duration="1.5449732915485583" />
<iteration index="73" Duration="1.6980736158816399" />
<iteration index="74" Duration="1.6219511082526878" />
<iteration index="75" Duration="1.5452583945734659" />
<iteration index="76" Duration="1.5272969040088356" />
<iteration index="77" Duration="1.6076959570109466" />
<iteration index="78" Duration="1.6915162463110391" />
<iteration index="79" Duration="1.6359211564690668" />
<iteration index="80" Duration="1.5270118009843827" />
<iteration index="81" Duration="2.0008530282502761" />
<iteration index="82" Duration="1.7191712397193442" />
<iteration index="83" Duration="1.5498200429710778" />
<iteration index="84" Duration="1.7984298806218249" />
<iteration index="85" Duration="1.6516018228344365" />
<iteration index="86" Duration="1.5506753520453458" />
<iteration index="87" Duration="1.8771183154744904" />
<iteration index="88" Duration="1.8685652247295366" />
<iteration index="89" Duration="1.5948663208937433" />
<iteration index="90" Duration="1.7077671187257693" />
<iteration index="91" Duration="1.5093354134451147" />
<iteration index="92" Duration="1.5307181403068171" />
<iteration index="93" Duration="1.5002121166508005" />
<iteration index="94" Duration="1.5441179824742903" />
<iteration index="95" Duration="1.627368065724113" />
<iteration index="96" Duration="1.6644314589520945" />
<iteration index="97" Duration="2.0273676095594055" />
<iteration index="98" Duration="1.7014948521796214" />
<iteration index="99" Duration="1.8865267152937122" />
<iteration index="100" Duration="1.5284373161084659" />
</iterations>
</performance>
</test>
<test name="System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: "")" type="System.Globalization.Tests.Perf_NumberCultureInfo" method="ToString" time="1.0029813" result="Pass">
<traits>
<trait name="Benchmark" value="true" />
</traits>
<performance runid="System.Globalization.Tests.dll-WindowsCore" etl="D:\tools\testing\relresults\11-16-2015\System.Globalization.Tests.dll-WindowsCore.etl">
<metrics>
<Duration displayName="Duration" unit="msec" />
</metrics>
<iterations>
<iteration index="0" Duration="1.6849588767395289" />
<iteration index="1" Duration="1.4982163954769021" />
<iteration index="2" Duration="1.6393423927670483" />
<iteration index="3" Duration="1.6955076886583811" />
<iteration index="4" Duration="1.4999270136258929" />
<iteration index="5" Duration="1.542122261300392" />
<iteration index="6" Duration="1.4982163954764474" />
<iteration index="7" Duration="1.4982163954764474" />
<iteration index="8" Duration="1.9603683987247678" />
<iteration index="9" Duration="2.1274387712742282" />
<iteration index="10" Duration="1.821523225632518" />
<iteration index="11" Duration="1.5908748785459466" />
<iteration index="12" Duration="1.5706325637834198" />
<iteration index="13" Duration="1.5683517395846138" />
<iteration index="14" Duration="1.7334263909606307" />
<iteration index="15" Duration="1.5478243217967247" />
<iteration index="16" Duration="1.7482517482517324" />
<iteration index="17" Duration="1.5338542735803458" />
<iteration index="18" Duration="1.5107609285691979" />
<iteration index="19" Duration="1.5635049881625491" />
<iteration index="20" Duration="1.5894493634218634" />
<iteration index="21" Duration="1.5270118009843827" />
<iteration index="22" Duration="1.5250160798109391" />
<iteration index="23" Duration="1.5076247952956692" />
<iteration index="24" Duration="2.0099763250450451" />
<iteration index="25" Duration="1.6276531687490206" />
<iteration index="26" Duration="1.5808962726773643" />
<iteration index="27" Duration="1.6424785260401222" />
<iteration index="28" Duration="1.5287224191333735" />
<iteration index="29" Duration="1.6963629977326491" />
<iteration index="30" Duration="1.5284373161084659" />
<iteration index="31" Duration="1.5968620420671868" />
<iteration index="32" Duration="1.5903046724961314" />
<iteration index="33" Duration="1.5666411214356231" />
<iteration index="34" Duration="1.7379880393577878" />
<iteration index="35" Duration="1.5190289162892441" />
<iteration index="36" Duration="1.5492498369208079" />
<iteration index="37" Duration="1.5572327216159465" />
<iteration index="38" Duration="1.5954365269431037" />
<iteration index="39" Duration="1.6792568162431962" />
<iteration index="40" Duration="1.6245170354759466" />
<iteration index="41" Duration="1.6937970705093903" />
<iteration index="42" Duration="1.8363485829236197" />
<iteration index="43" Duration="1.9831766407110081" />
<iteration index="44" Duration="1.7331412879361778" />
<iteration index="45" Duration="1.7493921603509079" />
<iteration index="46" Duration="2.2497479689259308" />
<iteration index="47" Duration="1.801566013894444" />
<iteration index="48" Duration="1.7827492142564552" />
<iteration index="49" Duration="1.7029203673041593" />
<iteration index="50" Duration="1.6216660052277803" />
<iteration index="51" Duration="1.6296488899229189" />
<iteration index="52" Duration="1.7000693370555382" />
<iteration index="53" Duration="1.522165049562318" />
<iteration index="54" Duration="1.5712027698327802" />
<iteration index="55" Duration="2.0142528704172946" />
<iteration index="56" Duration="1.5680666365597062" />
<iteration index="57" Duration="1.5572327216164012" />
<iteration index="58" Duration="1.5954365269435584" />
<iteration index="59" Duration="1.5446881885241055" />
<iteration index="60" Duration="1.5806111696524567" />
<iteration index="61" Duration="1.6772610950692979" />
<iteration index="62" Duration="1.523020358636586" />
<iteration index="63" Duration="2.0242314762858769" />
<iteration index="64" Duration="1.6584442954303995" />
<iteration index="65" Duration="1.515322576966355" />
<iteration index="66" Duration="1.5176034011651609" />
<iteration index="67" Duration="1.5307181403072718" />
<iteration index="68" Duration="1.6276531687490206" />
<iteration index="69" Duration="1.6228064173269559" />
<iteration index="70" Duration="1.6829631555660853" />
<iteration index="71" Duration="1.6079810600358542" />
<iteration index="72" Duration="1.7351370091096214" />
<iteration index="73" Duration="1.6775461980942055" />
<iteration index="74" Duration="1.650176307710808" />
<iteration index="75" Duration="1.7619366934436584" />
<iteration index="76" Duration="1.6878099069876953" />
<iteration index="77" Duration="1.7867406566037971" />
<iteration index="78" Duration="1.8086935895157694" />
<iteration index="79" Duration="1.7000693370555382" />
<iteration index="80" Duration="1.5828919938512627" />
<iteration index="81" Duration="1.8468973948424718" />
<iteration index="82" Duration="1.6228064173269559" />
<iteration index="83" Duration="1.6467550714123718" />
<iteration index="84" Duration="1.7733408144367786" />
<iteration index="85" Duration="1.6547379561079651" />
<iteration index="86" Duration="1.6855290827893441" />
<iteration index="87" Duration="1.6433338351143902" />
<iteration index="88" Duration="1.5828919938508079" />
<iteration index="89" Duration="1.5700623577336046" />
<iteration index="90" Duration="1.9985722040514702" />
<iteration index="91" Duration="1.7610813843693904" />
<iteration index="92" Duration="1.9201688722237122" />
<iteration index="93" Duration="1.9116157814787584" />
<iteration index="94" Duration="1.5891642603969558" />
<iteration index="95" Duration="1.8862416122688046" />
<iteration index="96" Duration="1.7191712397193442" />
<iteration index="97" Duration="1.9652151501463777" />
<iteration index="98" Duration="1.6541677500586047" />
<iteration index="99" Duration="1.5985726602166324" />
<iteration index="100" Duration="1.8788289336234811" />
</iterations>
</performance>
</test>
</collection>
@steveharter commented on Mon Nov 16 2015
Please assign to owner of PAL library, as this is not a globalization issue but instead the implementation of the PAL layer. The method _ecvt (\src\pal\src\cruntime\misctls.cpp) is taking up 95%+ of the time.
Note: the benchmarck test should probably “warm up” the current culture by performing a number.ToString(cultureInfo) before the start of the timing. This will force a one-time load of all of the localization information into the cultureinfo (like the character for the thousands separator, etc).
There are a few inefficiencies in _ecvt:
- Very large string buffer to hold the result (~350 chars)
- A copy of the entire string buffer due to the format string using exponent syntax %.348e" where the “e” is almost at the end of string typically with 100s of zeros (for small numbers).
- Re-writing sprintf_s or using an alternate Linux method may be more efficient. One inefficiency is that sprintf_s has a string-based format string which must be parsed
Changing the implementation to not use the large buffer makes the code run 4x+ faster, such as inserting this code at the beginning (sample - won’t work in all cases): pThread = InternalGetCurrentThread(); lpStartOfReturnBuffer = lpReturnBuffer = pThread->crtInfo.ECVTBuffer; sprintf_s(lpStartOfReturnBuffer, sizeof(TempBuffer), “%lf”, value); *dec = 1; *sign = 0; goto done;
@ianhays commented on Mon Nov 16 2015
Please assign to owner of PAL library, as this is not a globalization issue but instead the implementation of the PAL layer. The method _ecvt (\src\pal\src\cruntime\misctls.cpp) is taking up 95%+ of the time.
Thanks for looking into this Steve, will do. Know who that is by any chance?
Note: the benchmarck test should probably “warm up” the current culture by performing a number.ToString(cultureInfo) before the start of the timing. This will force a one-time load of all of the localization information into the cultureinfo (like the character for the thousands separator, etc).
This is taken into account in the analysis. The first few iterations are ignored.
@stephentoub commented on Tue Nov 17 2015
cc: @sergiy-k
@steveharter commented on Tue Nov 17 2015
Not sure who the owner(s) are or the original author. I am willing to take it, but it would have to come after rc2.
Another option is to measure the performance of using ICU to do that parsing.
@gkhurana commented on Wed Dec 09 2015
A few alternatives to sprintf_s, ecvt are snprintf and ecvt_r on unix. ecvt* family of functions are now obsolete and not recommended.
@krwq commented on Mon Nov 28 2016
@ianhays Could you please share the units of the measurements and how does that impact the whole framework? I think we should not be optimizing 1ns->40ns regression or really rarely used calls as it is simply not worth the effort. I will leave this open as up-for-grabs. Please change if you believe this should be fixed sooner.
@ianhays commented on Mon Dec 05 2016
The measurements were made using msbuild /t:buildandtest /p:Performance=true which calls the xunit.performance runner. It looks like this test has actually been removed since the measurement was made, so there isn’t any way to repro it in the same way.
really rarely used calls as it is simply not worth the effort.
I wrote a perf test for this method because usage data showed it high a relatively high utilization rate
I think we should not be optimizing 1ns->40ns regression
If you don’t feel it is a necessary optimization please feel free to close out the issue. These micro-measurements were done on thousands of iterations which is dissimilar to the actual usage, so whether it is worth improving is up to you.
@AndyAyersMS commented on Mon Dec 05 2016
There are quite a few low-level native helper routines that appear to be slower on linux, for instance much of the math library. See for instance some of the discussion and data in dotnet/coreclr#4847.
@karelz commented on Wed Mar 22 2017
The issue seems to be blocking https://github.com/dotnet/corefx/issues/16636 (which is blocking .NET Core adoption for a customer who reached to us via CSS). @mellinoe can you please help push it forward as part of your perf work in 2.0? (feel free to work with @krwq and @tarekgh, the area owners) See https://github.com/dotnet/corefx/issues/16636 for additional link to CoreCLR (maybe there are some dupes here).
@tannergooding commented on Wed Mar 22 2017
I have https://github.com/dotnet/coreclr/issues/9373, which is tracking the general perf issues with the math library on Linux.
@tarekgh commented on Wed Mar 22 2017
Looks to me there is no action from the library side on this one. all changes would be from the runtime side.
@karelz looks there are some tracking issues targeting fixing this issue. so I think we should close this one. I already ensured dotnet/coreclr#4847 and dotnet/coreclr#9373 are marked to v2.0.0
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Comments: 49 (47 by maintainers)
I have tried the CoreRT implementation and I got the number 2200 ms which is faster than the current implementation by 1.6 times (this almost the same result that @jkotas got).
considering the results, it is clear now that the way we go for Linux is to use ecvt_r. we can consider the corert implementation for OSX. still not perfect but would be acceptable for v2.0.
what people think about this plan?
I agree that for default formatting it’s not too bad that we don’t get identical output, but the way I would expect serialization/deserialization to work is to use the “R” (for roundtrip) format string, which should guarantee that absolutely identical FP values are produced after a serialization/deserialization cycle, which can be tested for with == without any allowance for epsilons. If we are not using “R” is our serialization code, I would consider that a bug in the serialization library. If we are using “R” but we don’t roundtrip FP values, I would consider that a bug in our FP printing code, regardless of what the IEEE spec says, because that’s what R is supposed to guarantee.
I have tested with Double.MaxValue/2 and got interesting results which confirm the plan. I run the test with same loop mentioned above.
Linux without any changes it took 13,200 ms Windows it took 6200 ms Linux with the corert implementation it took 3900 ms Linux with using ecvt_r 1100 ms
It is clear using ecvt_r is the best and corert implementation was much better than Windows in such case
Some update:
I have tried changing the implementation of _ecvrt to just call ecvt_r and I am seeing this is getting us back to Windows perf numbers (and even a little bit better).
I used the loop:
on Windows 10 VM I got this executed in average of 650 ms on Ubuntu VM (same Windows 10 VM configuration and running on the same host machine) I got 3600 ms On same Ubuntu VM with using ecvt_r I got 500 ms
I used Stopwatch but the results is consistent and the number mentioned here is the average.
I am going to try to use CoreRT implementation (I’ll convert it to native implementation for the sake of the experiment) and will get the perf numbers for it.