runtime: Internal CLR errors.

I am assuming (perhaps incorrectly), this is the correct place to report. Since upgrading to 6.0 I have noticed my program randomly (and not so randomly) crashing with internal CLR errors. These are always under fairly heavy memory usage. In some cases I can reproduce consistently. A typical example is:

Fatal error. Internal CLR error. (0x80131506)
Repeat 2 times:
--------------------------------
   at System.GC._Collect(Int32, Int32)
--------------------------------
   at System.GC.GetTotalMemory(Boolean)
  ....

(other times there is no stack trace at all - when this happens without me calling GetTotalMemory)

I have tried to rule out that this is a memory hardware problem on my machine by running the windows memory diagnostic which reports no errors. I have also reverted my projects back to 5.0 and the problem goes away. My programs heavily use BigInteger (which I notice has a bunch of unsafe code, but certainly can’t say that is the cause). I do not have any unsafe code of my own or any linked 3rd party libraries.

In any case I am reporting this so at least if you are getting similar reports from others then you know it is not just one person. Is there anything I can do that would help to further diagnose the problem? (assuming anyone on your side is interested in trying to track this down.) For example, I could send a bunch of .exe’s and .dll’s with a reproducible test case, or I could run under debugger (which may not help since the reproducible cases I have are all for release mode and this is somewhere deep inside .net)

dotnet --info is listed below. This is a .Net core command line application. (Although net6.0-windows build). I am using minimal windows functionality (console.writeline). Virtually only library classes I use are BigInteger, generic dictionaries, arrays, value-tuples, immutable collections and that is about it. I have 32GB installed, but this is not occurring at anywhere near that limit - in fact even below 500MB.

.NET SDK (reflecting any global.json):
 Version:   6.0.100
 Commit:    9e8b04bbff

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19042
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\6.0.100\

Host (useful for support):
  Version: 6.0.0
  Commit:  4822e3c3aa

.NET SDKs installed:
  5.0.402 [C:\Program Files\dotnet\sdk]
  6.0.100 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 3.1.20 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 3.1.20 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.20 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.11 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 21 (14 by maintainers)

Most upvoted comments

I’ve tested on 7.0 and this issue appears to be fixed. Thanks.

From: msftbot[bot] @.> Sent: August 16, 2022 8:02 PM To: dotnet/runtime @.> Cc: ahowroyd @.>; Mention @.> Subject: Re: [dotnet/runtime] Internal CLR errors. (Issue #64884)

This issue has been automatically marked no-recent-activity because it has not had any activity for 14 days. It will be closed if no further activity occurs within 14 more days. Any new comment (by anyone, not necessarily the author) will remove no-recent-activity.

— Reply to this email directly, view it on GitHubhttps://github.com/dotnet/runtime/issues/64884#issuecomment-1217402800, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AEAPZREG5DXB555HEPAVJETVZRI5LANCNFSM5NV3ECZA. You are receiving this because you were mentioned.Message ID: @.***>

ahowroyd on Aug 18, 2022

Stacktrace of the crash:

0:000> k
 # Child-SP          RetAddr               Call Site
00 (Inline Function) --------`--------     coreclr!MethodTable::GetFlag [D:\a\_work\1\s\src\coreclr\vm\methodtable.h @ 3845] 
01 (Inline Function) --------`--------     coreclr!MethodTable::HasComponentSize [D:\a\_work\1\s\src\coreclr\vm\methodtable.h @ 1728] 
02 (Inline Function) --------`--------     coreclr!WKS::my_get_size+0x4 [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 10683] 
03 (Inline Function) --------`--------     coreclr!WKS::gc_heap::find_next_marked+0x84 [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 27066] 
04 000000b1`e657c9d0 00007ff8`ef3502e3     coreclr!WKS::gc_heap::plan_phase+0x544 [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 28045] 
05 000000b1`e657cde0 00007ff8`ef34ebfd     coreclr!WKS::gc_heap::gc1+0xab [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 20092] 
07 000000b1`e657ce40 00007ff8`ef34f99b     coreclr!WKS::gc_heap::garbage_collect+0x1ad [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 21768] 
08 000000b1`e657ce90 00007ff8`ef4161b6     coreclr!WKS::GCHeap::GarbageCollectGeneration+0x14f [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 44700] 
09 000000b1`e657cee0 00007ff8`ef4162de     coreclr!WKS::GCHeap::GarbageCollect+0x86 [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 43887] 
0a 000000b1`e657cf20 00007ff8`8fa6be50     coreclr!GCInterface::Collect+0x5e [D:\a\_work\1\s\src\coreclr\vm\comutilnative.cpp @ 1007] 
0b 000000b1`e657cf80 00007ff8`8fa60fa3     System_Private_CoreLib!System.GC.GetTotalMemory+0x110 [/_/src/coreclr/System.Private.CoreLib/src/System/GC.cs @ 347]

jkotas on Feb 7, 2022

Scanning through the rest of your code, it’s likely that additional performance gains could be made in places - for instance, Factorials should be using an ImmutableArrayBuilder, if not a completely static/constant array (there are surprisingly few values in the 2^32 range, for instance). I don’t know enough of what you’re doing overall, though, to suggest more overall improvements.

Clockwork-Muse on Feb 7, 2022

CLRCrashSrc.zip net6.0-windows.zip

I have created a stripped down version of my program. Attached both source and binaries.

Steps to reproduce. Seems to require a target of windows. (I am running a relatively fast windows 10). Also most easily reproduced with a Release build. Debug also eventually crashes but takes much longer.

Run the program with the shown parameters: clrcrash 22 0 21 Rows 1…21 will not crash. Also if you start at 7 instead of 0 it doesn’t crash on 7 - it seems to require a little warm up, so I’d suggest at least in the first instance of trying the parameters exactly as given (just to see if you even encounter the issue). Failure can be expected after the 7-th or 8-th term (about half an hour to one hour; the full run to 22 terms is several more hours). For me it consistently crashes eventually - but its fussy - so not entirely systematic.

Two sample runs are shown below:

 C:\Work\CLRCrash\bin\Release\net6.0-windows>clrcrash 22 0 21
1
1
358178
52638543270855341
496661499687277811428895288
304368202894638666212732881784537824
16236558354165671130857683402154320827770366
Fatal error. Internal CLR error. (0x80131506)

C:\Work\CLRCrash\bin\Release\net6.0-windows>clrcrash 22 0 21
1
1
358178
52638543270855341
496661499687277811428895288
304368202894638666212732881784537824
16236558354165671130857683402154320827770366
92649048206788461704931591250082335023644515566432
Fatal error. Internal CLR error. (0x80131506)
Repeat 2 times:
--------------------------------
   at System.GC._Collect(Int32, Int32)
--------------------------------
   at System.GC.GetTotalMemory(Boolean)
   at CLRCrash.GraphByDegreeSeq+Processor.CheckMem()
   at CLRCrash.GraphByDegreeSeq+Processor.SaveCanonical(Int32, System.Numerics.BigInteger)
   at CLRCrash.GraphByDegreeSeq+Processor.ExploreGraph(Int32, Int32, System.Numerics.BigInteger, Int32)

ahowroyd on Feb 7, 2022