efcore: The instance of entity type cannot be tracked because another instance with the same key value for { 'ID'} is already being tracked.

// https://stackoverflow.com/questions/62253837/the-instance-of-entity-type-cannot-be-tracked-because-another-instance-with-the // https://github.com/dotnet/efcore/issues/12459 // https://docs.microsoft.com/en-us/ef/core/change-tracking/identity-resolution

.net 6.0 EF core 6.0.8 Project attached.

As commented in the code the problem I am having is discovering if “another instance” even exists and if it does where the mysterious “another instance” is. If “another instance” does in fact exist, why is it not being assigned to series in the line below?:

    // Next line we set series to the instance of series being tracked by the db.
    // If there is "another instance", series should be set
    // to that "another instance" since we are directly assigning it:

    series = db.Series.First(x => x.Symbol == symbol);

I do not see any logical path forward to save series to disk. In the space of two lines of code I am reading an entity from disk and trying to write it back and am unable to do so. How do I modify series and save it to disk?

using Microsoft.EntityFrameworkCore;

public class Program
{
	private const string connectionString = "Data Source=.;Initial Catalog=EFDupeTest;Integrated Security=True;MultipleActiveResultSets=True;";
	private Db db;

	public Program(Db db) => this.db = db;


	public static async Task Main(string[] args)
	{
		DbContextOptionsBuilder builder = new DbContextOptionsBuilder();
		builder.UseSqlServer(connectionString);
		Db db = new Db(builder.Options);
		db.Database.EnsureDeleted();
		db.Database.EnsureCreated();
		await new Program(db).DownloadSeriesRelease("xyz");
	}

	public async Task DownloadSeriesRelease(string symbol)
	{
		Series? series = db.Series.FirstOrDefault(x => x.Symbol == symbol);
		
		if (series == null)
			await DownloadSeries(symbol); 

		// Next line we set series to the instance of series being tracked by the db.
		// If there is "another instance", series should be set
		// to that "another instance" since we are directly assigning it:

		series = db.Series.First(x => x.Symbol == symbol);
		
		//
		// series has been saved to disk and now has an ID of 1.
		// In real code we modify a property of series here and save to disk.
		// How do we save series to disk?
		//


		// Error here
		// System.InvalidOperationException
		// The instance of entity type 'Series' cannot be tracked because another instance with the same key
		// value for { 'ID'} is already being tracked.When attaching existing entities,
		// ensure that only one entity instance with a given key value is attached.
		// Consider using 'DbContextOptionsBuilder.EnableSensitiveDataLogging' to see the conflicting key values.

		db.Entry(series).State = series.ID == 0 ? EntityState.Added : EntityState.Modified;
		await db.SaveChangesAsync();
	}

	public async Task DownloadSeries(string symbol)
	{
		Series series = new Series { Symbol = symbol };  
		await SaveSeries(series);
	}


	public async Task SaveSeries(Series series)
	{
		Series? dupe = await db.Series.FirstOrDefaultAsync(x => x.ID != series.ID && x.Symbol == series.Symbol);
		db.Entry(series).State = series.ID == 0 ? EntityState.Added : EntityState.Modified;
		await db.SaveChangesAsync();
	}

}

public class Series
{
	public int ID { get; set; }
	public string Symbol { get; set; }
}

public class Db : DbContext
{
	public DbSet<Series> Series { get; set; }

	public Db(DbContextOptions options) : base(options)
	{
		InitalizeContext();
	}

	protected virtual void InitalizeContext()
	{
		// https://blog.oneunicorn.com/2012/03/12/secrets-of-detectchanges-part-3-switching-off-automatic-detectchanges/
		ChangeTracker.AutoDetectChangesEnabled = false;
		ChangeTracker.QueryTrackingBehavior = QueryTrackingBehavior.NoTracking;
		Database.SetCommandTimeout(360);
	}
}

EFDupeTest.zip

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 44 (22 by maintainers)

Most upvoted comments

@smitpatel I don’t need your remarks.

@sam-wheat The code first creates and saves a Series instance. This instance is now being tracked by the context. The code then queries again using no-tracking. This creates a new instance of the same entity. The code then attempts to track this duplicate instance.

why is it not being assigned to series in the line below?:

Because the query has tracking disabled.

I do not see any logical path forward to save series to disk.

It’s not clear to me why the code is using a no-tracking query for this. The code doesn’t throw if tracking is used appropriately. You might want to read up on the basics of change tracking.

@roji

As stated previously, I provided that snippet to demonstrate what my code does NOT do. I documented that clearly in the code.

In my original post please identify the specific line where I instruct EF to start tracking a different instance with the same ID.

Please do not close this ticket until this question is answered. Thank you.

@ajcvickers

Because there is no other instance of series.

This line in DownloadSeriesRelease is the first and only place where series is assigned a non null value. And that value per the documentation I cited is not tracked. There are no other instances of series.

series = db.Series.First(x => x.Symbol == symbol); // not tracked

@ajcvickers addressed things above, but as I have an in-progress comment, I’ll post it anyway…

QueryTrackingBehavior.NoTracking tells EF to not track. Calling Entry(…) API tells EF to track. To satisfy both EF creates a duplicate entity - the effect of which, as indicated by this ticket, creates a minefield.

This is where your understanding of EF’s API contracts/obligations is off. QueryTrackingBehavior.NoTracking only means that queries don’t track - note the “Query” part in the enum name. The Entry() is a completely separate API which has nothing to do with querying, and therefore isn’t affected by that setting. You can use Entry() to start tracking an arbitrary instance, regardless of whether it came from an EF query, instantiated directly, or whatever.

Allow QueryTrackingBehavior.NoTracking to live up to it’s promise to not track any of the entities that are returned from a LINQ query.

As written above, that is exactly what QueryTrackingBehavior.NoTracking does. But you’re using an entirely separate API (Entry) to very explicitly tell EF to start tracking the untracked instance returned from your non-tracking query.

So there are no contradictory obligations here, just two different APIs - querying and Entry().

I’m again guessing that you expect EF to stop tracking the 1st instance when your code returns from DownloadSeries (since it “goes out of scope”, or since SaveChangesAsync has been called)

Should I expect something else? I expect EF to expose APIs that allow me to to use it in a consistent manner. […] Entry(…) provides a mechanism to track but it has no natural end to its lifetime that EF can manage.

Here again, it seems like you have certain assumptions/expectations which simply don’t correspond to how EF works. To quote from the introductory docs on change tracking:

Entity instances are no longer tracked when:

  • The DbContext is disposed
  • The change tracker is cleared (EF Core 5.0 and later)
  • The entities are explicitly detached

Note that as we’ve written above, SaveChanges does not cause anything to stop getting tracked. Among other things, there are valid scenarios where one may want to continue tracking after SaveChanges, so clearing at that point would make those scenarios impossible.

I think it’s fine to question to EF’s design decisions here, even if there’s little chance we’d change this (e.g. because of the breaking change). But I highly recommend first understanding the EF way of doing things; in this case, that would be via a plain, regular tracking query. Once you have a firm grasp on how EF works, you’d at the very least be in a better place to understand why some of these decisions are made.

@sam-wheat

Why is it that by following a very normal pattern of retrieving data I can cause EF to commit this awful deed?

This is not a “normal pattern”. In fact, it is explicitly called out in the docs as an anti-pattern–see https://docs.microsoft.com/en-us/ef/core/change-tracking/explicit-tracking#introduction, which states “Attaching entities to the same DbContext instance that they were queried from should not normally be needed. Do not routinely perform a no-tracking query and then attach the returned entities to the same context. This will be slower than using a tracking query, and may also result in issues such as missing shadow property values, making it harder to get right.”

What pattern can I follow that will prevent this and that will not enlist me in the business of managing tracking status on behalf of EF?

The correct thing to do, as I indicated in the very first comment on this thread, is to do a tracking query here. You will then get back the tracked entity instance.

Should I expect something else? I expect EF to expose APIs that allow me to to use it in a consistent manner.

You need to learn how to use the APIs in the manner they were intended to be used. This is why we have documentation.

I think of tracking in a manner similar to locking a table in sql server. Sometimes it must be done but in all cases the time spent doing it should be minimized.

This is not a good an analogy. The two things are not similar in any relevant way.

Entry(…) provides a mechanism to track but it has no natural end to its lifetime

The natural life time is the unit-of-work, which should be associated with the context lifetime. Again, this is called out in the documentation.

Your suggestions reflect your expectations of how an ORM should work. Maybe they would result in a good ORM; maybe not. It’s not the way EF works and never will be.

@roji

I think the problem here is that EF has contradictory obligations related to tracking. The byproduct of EF trying to simultaneously respect these mutually excusive requirements inflicts more harm than the good either obligation provides.

QueryTrackingBehavior.NoTracking tells EF to not track. Calling Entry(...) API tells EF to track. To satisfy both EF creates a duplicate entity - the effect of which, as indicated by this ticket, creates a minefield.

That means it returns a new, unrelated instance in series.

Now we are clear about how the dupe is being created and who is responsible. Why is it that by following a very normal pattern of retrieving data I can cause EF to commit this awful deed? What pattern can I follow that will prevent this and that will not enlist me in the business of managing tracking status on behalf of EF? Fair questions!

I’m again guessing that you expect EF to stop tracking the 1st instance when your code returns from DownloadSeries (since it “goes out of scope”, or since SaveChangesAsync has been called)

Should I expect something else? I expect EF to expose APIs that allow me to to use it in a consistent manner.

I think of tracking in a manner similar to locking a table in sql server. Sometimes it must be done but in all cases the time spent doing it should be minimized. Entry(..) provides a mechanism to track but it has no natural end to its lifetime that EF can manage. Clearing tracking flag when SaveChanges is called will allow for a pattern that will:

  • Limit tracking lifetime to two lines of code that are naturally contiguous.

  • Preclude the need for EF to create a dupe when returning an entity.

  • Allow EF to return objects that are in a consistent state.

  • Allow the developer to offload the task of managing tracking state to EF.

  • Allow QueryTrackingBehavior.NoTracking to live up to it’s promise to not track any of the entities that are returned from a LINQ query.

      // assume `QueryTrackingBehavior.NoTracking`
      public void ChangeSomeData()
      {
      	Entity entity = db.Entities.FirstOrDefault(...);
    
      	if(entity is null)
      	{
      		bool downloaded = DownloadFromVendor(); 
      		
      		if(downloaded)
      			entity = db.Entities.FirstOrDefault(...);
      		else
      			return;
      	}
      	// minefield here. Is entity tracked? Why should the developer need to check. 
      }
      private bool DownloadFromVendor()
      {
      	Entity e = httpClient.Get(...);
      	db.Entry(e).State = EntityState.Added;  // start tracking
      	SaveChanges(); // proposed - look at `ChangeTracker.QueryTrackingBehavior` and stop tracking if `QueryTrackingBehavior.NoTracking`
      }
    

@sam-wheat we’re done that above as well, but here goes once more. Looking at the code in the OP:

  1. DownloadSeries is called (since series == null). It creates a new instance of Series and saves it to the database. After that completes and DownloadSeries returns, the context is tracking that instance (even if the variable is out of scope).
  2. You re-query for the series (series = db.Series.First(x => x.Symbol == symbol)). Since the context is configured with AutoDetectChangesEnabled=false, this is a non-tracking query which bypasses the change tracker. That means it returns a new, unrelated instance in series. This in itself isn’t a problem, but note that the earlier instance (created and saved inside DownloadSeries) is still tracked by the context.
  3. You now call db.Entry(series) (2nd before last), passing it the untracked instance from step 2 above. This explicitly asks EF to start tracking the instance, but the earlier instance is still tracked, and has the same ID. Therefore, EF raises an exception.

I’m again guessing that you expect EF to stop tracking the 1st instance when your code returns from DownloadSeries (since it “goes out of scope”, or since SaveChangesAsync has been called) - this is not the case. I highly suggest stepping through the code with your debugger, line-by-line, and examining the ChangeTracker property on the context (see @ajcvickers’s screenshots above). This tells you unambiguously what is currently tracked by the context.

Once all of the above is clear and you have the right mental model of change tracking, we can suggest an alternative code pattern to do what you’re trying to achieve. But it’s important to first understand why your current code doesn’t work.

@sam-wheat we’ve done that several times already, but looking at your sample in https://github.com/dotnet/efcore/issues/28822#issuecomment-1222576059

1 Series firstSeries = db.Series.FirstOrDefault(x => x.Symbol == "xyz"); // null
2 Series secondSeries = new Series { Symbol = "xyz" };  // in call to DownloadSeriesRelease this goes out of scope
3 Series? dupe = await db.Series.FirstOrDefaultAsync(x => x.ID != secondSeries.ID && x.Symbol == secondSeries.Symbol); // null
4 db.Entry(secondSeries).State = EntityState.Added;
5 await db.SaveChangesAsync();
6 firstSeries = db.Series.FirstOrDefault(x => x.Symbol == "xyz"); // should not be tracked see QueryTrackingBehavior.NoTracking;
7 db.Entry(firstSeries).State = EntityState.Modified; // crash here
  1. You instantiate a new Series on line 2, assigning it to secondSeries.
  2. You instruct EF to start tracking secondSeries with state “Added” on line 4 (db.Entry)
  3. You load a different instance (firstSeries) via a non-tracking query on line 6. At this point firstSeries isn’t tracked because the query has NoTracking
  4. You instruct EF to start tracking firstSeries on line 7 (db.Entry). Since secondSeries is already being tracked (step 2 above), you get an exception.

In other words, db.Entry is an API which tells EF “start tracking this”.

@roji

You then try instruct EF to start tracking a different instance with the same ID,

Kindly refer me to the line in the example I’ve provided where I do this.

EF Core doesn’t stop tracking entities just because you called SaveChanges.

What good, than, is QueryTrackingBehavior.NoTracking? This behavior is self-defeating.

I think you are assuming that because no-tracking queries don’t track the instances they return, then the context will never track entities through some other mechanism.

I don’t make that assumption at all. I am guided by a specific section of documentation which I have cited many times and will cite again:

// The change tracker will not track any of the entities that are returned from a LINQ query.

I read this to mean that I should be able run the query shown below with complete confidence that the entity (entities) I receive is/are not tracked. This seems very clear to me.

series = db.Series.First(x => x.Symbol == symbol);  // not tracked

The second time it fails because the instance from the first call is still being tracked.

Do you mean to say that the entity is referenced by by the dbcontext, which would prevent it from being garbage collected? That seems reasonable. As for tracking behavior, here are the events that I expect to occur:

  • DownloadSeries - new up an entity.
  • SaveSeries - Set the state to Modified, thus telling EF to track the entity.
  • SaveSeries - Call SaveChanges(). At this point the entity should revert to being untracked. EF’s work is done.
  • Control returns to DownloadSeriesRelease where I assign new entity to series. At this point I should be able to have full faith and confidence that the entity returned as a result of my LINQ query is not tracked. This is what QueryTrackingBehavior.NoTracking; assures me will happen but appears to not happen.
Series s = null;
dbContext.Series = new Series { Symbol = "xyz" };
s = dbContext.Series;

How many instances of series are there?

@ajcvickers

lol Arthur did you read what you just wrote?

All your queries are no-tracking and none of them result in an instance being tracked.

Ok nothing is tracked…

It isn’t discarded. It’s tracked.

Oh wait…

It doesn’t. It would if you were using a tracking query.

What? I’m not using a tracking query??

There is one instance being tracked, and once instance not being tracked.

Ok lets just take the average… lolol

@ajcvickers

It’s because this instance is tracked that the call later in the code to track the second instance fails. In other words, track one instance with a given key–okay. Track another instance with the same key–fail.

There is only one instance. Furthermore, this directly contradicts the documentation for QueryTrackingBehavior.NoTracking. The documentation fairly describes the behavior I expect from the setting :

// The change tracker will not track any of the entities that are returned from a LINQ query. If the /// entity instances are modified, this will not be detected by the change tracker and /// will not persist those changes to the database.

Here is the behavior I expect to happen AFTER ‘SaveChanges’ is called in ‘SaveSeries’:

The method returns to DownloadSeries and then to DownloadSeriesRelease. The local instance of series that was created in DownloadSeries is disposed. The next line of code that executes is this one:

series = db.Series.First(x => x.Symbol == symbol);  // not tracked

series, which was null prior to the call to DownloadSeries should be set to an untracked instance of Series. This is the place where QueryTrackingBehavior.NoTracking must be respected. There are no other instances of series that exist at this point. Even if the context returns the instance of series that was created in DownloadSeries, series is simply pointing to that instance. There is only one instance and it is not tracked.

This line in DownloadSeriesRelease should not fail. There is only one instance of series and it is not tracked:

db.Entry(series).State = series.ID == 0 ? EntityState.Added : EntityState.Modified;

Does that line throw an error? Why is it relevant?

The line that throws the error this one in DownloadSeriesRelease, where I retrieve an untracked entity from the context and attempt to set it’s state to added:

 series = db.Series.First(x => x.Symbol == symbol); // not tracked
db.Entry(series).State = series.ID == 0 ? EntityState.Added : EntityState.Modified; // throws

Where you track the first instance?

I don’t track the first instance. Line 66 is the method declaration for DownloadSeries. Is there another place in the code where I track series? Please cite the line.

See that the first instance is still tracked before attempting to track the second instance by setting its state to Modified.

What is the specific line number in my code where I do that?

@ajcvickers

// Next line we set series to the instance of series being tracked by the db.

This comment in my code is actually wrong. The instance we are getting is untracked per previously cited documentation related to QueryTrackingBehavior.NoTracking.

So, bottom line, I have a single instance of series which is not tracked and EF is crashing telling me there is another tracked instance.

@ajcvickers

The instance is tracked when you set the state to Added.

I can’t set the state to Added. EF crashes. That is the problem.

Did you try running my code? Please run the code in the project or in my original post.

@ajcvickers please erase that modification and use the code I originally posted. I added that subsequent code to show how my code is different. I also commented “in call to DownloadSeriesRelease this goes out of scope”:

Series secondSeries = new Series { Symbol = "xyz" };  // in call to DownloadSeriesRelease this goes out of scope

@ajcvickers

But as soon as you set the state of the entity with db.Entry(foo).State, then EF starts tracking it.

It does not. It throws the error I cited in my code.

Hopefully we agree that it should start tracking and we have made some progress.

@ajcvickers

This line tracks it:

In my code the second instance of series that is created in DownloadSeries goes out of scope. In DownloadSeriesRelease I assign the result of a linq query to my single instance of series. Per this documentation for QueryTrackingBehavior.NoTracking that instance is not tracked:

// The change tracker will not track any of the entities that are returned from a LINQ query. If the /// entity instances are modified, this will not be detected by the change tracker and /// will not persist those changes to the database.

series = db.Series.First(x => x.Symbol == symbol);  // not tracked

The code doesn’t throw if tracking is used appropriately.

Please explain how this code should be written assuming QueryTrackingBehavior.NoTracking is used as default behavior. You have referred me to the documentation several times - I do not understand how to apply it to my code. Also, the documentation I have cited appears to support my use case. How am I intrepetting it incorrectly?

Thanks.

@ajcvickers To simplify things a bit I modified Main as shown below. The lines between the test 1 comments follow the general sequence of my code however the secondSeries variable does not go out of scope as it does in the call to DownloadSeriesRelease.

public static async Task Main(string[] args)
{
    DbContextOptionsBuilder builder = new DbContextOptionsBuilder();
    builder.UseSqlServer(connectionString);
    Db db = new Db(builder.Options);
    db.Database.EnsureDeleted();
    db.Database.EnsureCreated();

    // test 1
    Series firstSeries = db.Series.FirstOrDefault(x => x.Symbol == "xyz"); // null
    Series secondSeries = new Series { Symbol = "xyz" };  // in call to DownloadSeriesRelease this goes out of scope
    Series? dupe = await db.Series.FirstOrDefaultAsync(x => x.ID != secondSeries.ID && x.Symbol == secondSeries.Symbol); // null
    db.Entry(secondSeries).State = EntityState.Added;
    await db.SaveChangesAsync();
    firstSeries = db.Series.FirstOrDefault(x => x.Symbol == "xyz"); // should not be tracked see QueryTrackingBehavior.NoTracking;
    db.Entry(firstSeries).State = EntityState.Modified; // crash here
    // test 1

    await new Program(db).DownloadSeriesRelease("xyz");
}

In my code the instance of series created in DownloadSeries goes out of scope. However the error message is telling me “another instance” exists. Is the actual error that the instance I am trying to track is already tracked? If so, why is it tracked? The following, from QueryTrackingBehavior.NoTracking intellisense is true, correct?;

// The change tracker will not track any of the entities that are returned from a LINQ query. If the /// entity instances are modified, this will not be detected by the change tracker and /// <see cref="DbContext.SaveChanges()" /> will not persist those changes to the database.

Hi @ajcvickers ,

The code first creates and saves a Series instance.

Which line of code are you referring to? I believe the first step is to create a dbcontext with tracking disabled.

It’s not clear to me why the code is using a no-tracking query for this. The code doesn’t throw if tracking is used appropriately.

I’ve been using the settings shown for several years now and I have not run into this issue. Note the link in the InitalizeContext method in the code.