efcore: OutOfMemoryException when calling SaveChanges
I get an OOM when saving LOTS of data (in a loop) to the DB with EF Core 2.2.6:
System.OutOfMemoryException: Exception of type ‘System.OutOfMemoryException’ was thrown.
at System.Text.StringBuilder…ctor(String value, Int32 startIndex, Int32 length, Int32 capacity)
at Microsoft.EntityFrameworkCore.Metadata.Internal.ConstraintNamer.Truncate(String name, Nullable1 uniquifier, Int32 maxLength) at Microsoft.EntityFrameworkCore.Metadata.Internal.ConstraintNamer.GetDefaultName(IProperty property) at Microsoft.EntityFrameworkCore.Metadata.RelationalPropertyAnnotations.get_ColumnName() at Microsoft.EntityFrameworkCore.Update.ColumnModification..ctor(IUpdateEntry entry, IProperty property, IRelationalPropertyAnnotations propertyAnnotations, Func1 generateParameterName, Boolean isRead, Boolean isWrite, Boolean isKey, Boolean isCondition, Boolean isConcurrencyToken)
at Microsoft.EntityFrameworkCore.Update.ModificationCommand.GenerateColumnModifications()
at Microsoft.EntityFrameworkCore.Update.ModificationCommand.<>c.<get_ColumnModifications>b__19_0(ModificationCommand command)
at Microsoft.EntityFrameworkCore.Internal.NonCapturingLazyInitializer.EnsureInitialized[TParam,TValue](TValue& target, TParam param, Func2 valueFactory) at Microsoft.EntityFrameworkCore.Update.ModificationCommand.get_ColumnModifications() at Microsoft.EntityFrameworkCore.Update.Internal.CommandBatchPreparer.CreateKeyValuePredecessorMap(Graph1 commandGraph)
at Microsoft.EntityFrameworkCore.Update.Internal.CommandBatchPreparer.TopologicalSort(IEnumerable1 commands) at Microsoft.EntityFrameworkCore.Update.Internal.CommandBatchPreparer.<BatchCommands>d__14.MoveNext() at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.Execute(DbContext _, ValueTuple2 parameters)
at Microsoft.EntityFrameworkCore.Storage.Internal.NoopExecutionStrategy.Execute[TState,TResult](TState state, Func3 operation, Func3 verifySucceeded)
at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.Execute(IEnumerable1 commandBatches, IRelationalConnection connection) at Microsoft.EntityFrameworkCore.Storage.RelationalDatabase.SaveChanges(IReadOnlyList1 entries)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(IReadOnlyList`1 entriesToSave)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(Boolean acceptAllChangesOnSuccess)
at Microsoft.EntityFrameworkCore.DbContext.SaveChanges(Boolean acceptAllChangesOnSuccess)
at Microsoft.EntityFrameworkCore.DbContext.SaveChanges()
The loop basically news up 1 DbContext and keeps adding entities to it. The DB (SQLite) is brand new and empty. The SaveChanges() method is called every 250 entities added. After several tens of thousands the OOM appears.
Ideas?
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 22 (10 by maintainers)
@adrianhara Maybe you should post a repro project?
@ajcvickers I’m sorry, I forgot to say that before starting the loop and after newing up the DbContext I do
dbContext.Database.BeginTransaction()and after the loop I commit it. I guess that means that no other transactions are created. So, let me restate my case/need:SaveChanges()every 250 entities added (can be changed to whatever number, I just don’t think I can call it at the very end, after adding all entities, because we’re talking about tens of thousands potentially)Any idea why this would throw an OOM? As the exception comes from StringBuilder I assume it has something to do with memory fragmentation going on when so many entities are added one after the other. Is this assumption correct? Is there anything I can do about it?
The inserts need to run transactionally that’s why I used 1 DbContext for everything. I guess I could start a transaction manually and commit it at the end, but it feels hackish…
Dispose the context at intervals?
@adrianhara sounds like you need bulk insert. See here : https://github.com/borisdj/EFCore.BulkExtensions
Sounds like the objects are not being disposed. Maybe you can show the code? it will be much easier 😃