efcore: Open DataReader error on a relatively simple scenario, but only under load in production

My client is getting this fairly frequently in production, when there are multiple concurrent users, but we can’t isolate and reproduce the problem in a dev environment. As far as we can tell the environments are identical (except for volume of requests). The app is an ASP.NET Core REST API using EF Core 2.2, and only two controller methods are showing the error:

There is already an open DataReader associated with this Command which must be closed first. at System.Data.SqlClient.SqlInternalConnectionTds.ValidateConnectionForExecute(SqlCommand command)

The context is being injected into the controller (Scoped). We have not (yet) tried to turn on MARS in production.

The simpler of the two methods goes something like this:

try {
  var es = ctx.Database.CreateExecutionStrategy();
  es.Execute(ctx => ectx =>
  {
    using (var txn = ectx.Database.BeginTransaction())
    {
      foreach (Thing t in ectx.Things.Where(criteria).ToList())
      {
        t.Prop = null;
        log ();
      }

      var t2 = ectx.Things.Include(t => t.OtherThing)
        .ThenInclude(o => o.AnotherThing)
        .ThenInclude(y => y.YetAnotherThing)
        .Include(t => t.OtherThing)
        .ThenInclude(o => o.SomeThing)
        .ThenInclude(s => s.FinalThing)
        .FirstOrDefault(criteria);
      if (t2 != null)
      {
        t2.Prop = "value";
      }

      log ();

      ectx.SaveChanges();
      txn.Commit();
    }
  }
}
catch (Exception ex) {
  log ();
  throw;
}

The logging that’s happening above is to a table in the same database, but through a different context.

We’re logging the generated SQL in our dev environments and it doesn’t appear that any of the queries are attempting to lazy load or otherwise open a second DataReader.

I’m less familiar with EF (mostly used NHibernate), but a little bit of research suggests to me that the retry strategy and explicit transaction here is unnecessary (since we’re only saving once at the end), but we left it in place in production until we can isolate and reproduce it (and better understand what’s going on).

Any advice on what else we can try to reproduce this in isolation, or where else we might look to detect environmental differences that could be causing it? Is MARS basically mandatory for EF Core, and what kind of other issues might we expect to see if we enable it?

Further technical details

EF Core version: 2.2 Database Provider: Microsoft.EntityFrameworkCore.SqlServer Operating system: Windows Server IDE: Visual Studio 2017

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 19 (10 by maintainers)

Most upvoted comments

Best option would be to create self-containing repro rather than trying to put something in our testbase. Our tests have a very interesting infrastructure setup to reuse as many components as possible to run tests faster on SqlServer. Which brings limitation like above that you cannot run query in multiple threads. Jumping through those hoops trying to figure out limitations may become time expensive.

That is expected. The TestBase uses DbConnection to configure SqlServer which is shared across all DbContext instances. If MARS is off then only one open DataReader can be present. When you run 1 query in 1 context, we will do buffer accordingly. But you are running them in parallel so even though you have created new DbContext for each thread, you are still using same connection which EF cannot know about and throws exception. That test failing is by design.

I threw it into \test\EFCore.Specification.Tests\Query\IncludeTestBase.cs, and the override in test\EFCore.SqlServer.FunctionalTests\Query\IncludeSqlServerTest.cs