runtime: Reasons for OutOfMemoryException
Here are the reasons I’m aware of that a .NET process could raise an OutOfMemoryException:
- Explicit throw of
OutOfMemoryExceptionby developer code (like Bitmap.Clone). - Allocation of a single object that exceeds inherent CLR limitations. For instance
new string('c', int.MaxValue) - A 32 bit process actually running out of virtual address space to use and failing on a simple
newoperation.
I’m sure this is not an exhaustive list though. I’m curious what the other reasons are that a process could OOM in .NET. Is there a good list somewhere I can read through?
Asking because recently I’ve been looking into a number of OOM bugs for Roslyn and I’m having problems identifying the cause. Specifically we’re seeing the following patterns:
- OOM running VBCSCompiler. Given this is a 64 bit process it’s hard to understand why this would generate an OOM.
- Running our unit tests for dotnet/roslyn. They will fail with an
OutOfMemoryException. There is no pattern to which test causes the OOM and it happens infrequently enough that we don’t have good historical data on it (this is changing going forward).
Hoping that learning more about OutOfMemoryException will give me new insight into these issues.
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 7
- Comments: 17 (15 by maintainers)
Also, on Windows its quite possible to OOM on running out of commit space. In general, when memory is allocated on Windows using VirtualAlloc, memory isn’t actually physically assigned when allocated, but instead when the memory is first touched, the OS will assign memory to the application, and the memory usage number will go up. However, even before the memory is touched, after a VirtualAlloc to commit memory, the committed memory is increased. If that committed memory number ever exceeds the amount of physical RAM + swap space on the machine, then the OS will treat it the same as running out of physical memory, and an OOM will be triggered. This is most commonly a problem on machines where the number of threads is high relative to the amount of RAM+swap, or if swap is disable as each thread in Windows commits by default 1MB.
As an example, as I’m typing now, I have an MicrosoftEdgeCP.exe that is using 59,276KB of RAM (private working set), but has 234,908KB of commit in use.
There is one special OOM that seems to be somewhat unknown. It has a different text than the usual OOM: “Insufficient memory within specified address space range to continue the execution of the program.”
Some of it is documented in this KB (for which I had to do some arm wrestling to get published, including writing most of the content): https://support.microsoft.com/kb/3152158
Edit: The content seem to have been heavily modified since then.
Short story: The late-bound jumps in an assembly are +/- 2 GB (signed 32-bit) relative jumps. If the jump has to be further an 64-bit absolute jump trampoline is created. But if the CLR can’t find space for the trampoline within +/- 2 GB we get this OOM from the CLR.
!ao would show you whether you have a commit failure if you are getting OOM due to an allocation on the GC heap, or if the GC has trouble getting memory for its own bookkeeping purpose.
for other scenarios where you might get OOM like from native allocations, “!address -summary” which will show you how much memory is already committed.
when you get an OOM from a managed process, the first and foremost thing to identify is whether you are getting it from a managed allocation or a native allocation. !ao will tell you. you could set your post mortem debugging to run this sos command (and better yet, capture a dump if it’s not easy to repro).
you can of course get OOM from running out of physical storage, VA is not the only limiting factor when you need more memory.
over the years I fixed a few premature OOM bugs in GC (mostly in Server GC which VBCSCompiler does use I think last I heard); I haven’t seen any in a while.
another thing that can help is to set this env var:
COMPlus_GCBreakOnOOMto 1 which will do a DebugBreak() as soon as GC observes an OOM. so you don’t end up running a lot of stuff (which may very well change the memory situation) before you actually get to process the OOM exception.On 32 bit platforms, the OOMs due to limited VA space can also happen when too many threadpool threads are created. Each thread allocates VA space for its stack and multiplied by the number of threads, it can eventually consume the whole available VA space. I was looking into such case on ARM some time ago too.
Another reason for OOM that I’ve seen recently is getting out of open files limit on Unix. That is also reported as out of memory. Sockets count against that limit too. On my Ubuntu 16.04, the default limit is 1024 files. See https://github.com/dotnet/coreclr/issues/25645 for an example of such OOM.