efcore: Including collection navigation after optional navigation throws NRE with async in 2.0.0

We are trying to upgrade our MySQL provider to 2.0.0. Our Discriminator Test that tests TPH Inheritance is resulting in this NullReferenceException

Here is the Discriminator Test - read it for a good laugh, I’m put some time into making sure the Hierarchies were fun 😄

Here is the Query that is causing the Exception:

				var teachers = await db.People.OfType<PersonTeacher>()
					.Where(m => m.Id >= _fixture.LowestTeacherId && m.Id <= _fixture.HighestTeacherId)
					.OrderBy(m => m.Id)
					.Include(m => m.Family)
					.ThenInclude(m => m.Members)
					.Include(m => m.Students)
					.ThenInclude(m => m.Family)
					.ThenInclude(m => m.Members)
					.ToListAsync();

Here is the Exception:

Failed   Pomelo.EntityFrameworkCore.MySql.PerfTests.Tests.Models.DiscriminatorTest.TestDiscriminator
Error Message:
 System.NullReferenceException : Object reference not set to an instance of an object.
Stack Trace:
   at Microsoft.EntityFrameworkCore.Query.Internal.IncludeCompiler.IncludeLoadTreeNodeBase.<_AwaitMany>d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.EntityFrameworkCore.Query.Internal.IncludeCompiler.<_IncludeAsync>d__18`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.EntityFrameworkCore.Query.ExpressionVisitors.Internal.TaskLiftingExpressionVisitor.<_ExecuteAsync>d__8`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.AsyncSelectEnumerable`2.AsyncSelectEnumerator.<MoveNext>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at System.Linq.AsyncEnumerable.SelectEnumerableAsyncIterator`2.<MoveNextCore>d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at System.Linq.AsyncEnumerable.AsyncIterator`1.<MoveNext>d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.EntityFrameworkCore.Query.Internal.AsyncLinqOperatorProvider.ExceptionInterceptor`1.EnumeratorExceptionInterceptor.<MoveNext>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at System.Linq.AsyncEnumerable.<Aggregate_>d__6`3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Pomelo.EntityFrameworkCore.MySql.PerfTests.Tests.Models.DiscriminatorTest.<TestDiscriminator>d__2.MoveNext() in /home/caleb/Projects/caleb/Pomelo.EntityFrameworkCore.Mysql/test/EFCore.MySql.FunctionalTests/Tests/Models/DiscriminatorTest.cs:line 96
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

About this issue

Original URL
State: closed
Created 7 years ago
Reactions: 5
Comments: 46 (28 by maintainers)

Commits related to this issue

Fix to #9038 - NullReferenceException when using TPH Inheritance Problem was that for the fixup for _IncludeAsync method would sometimes return default(Task) (i.e. null) if there was nothing to inclu... — committed to dotnet/efcore by maumar 7 years ago
Fix to #9038 - NullReferenceException when using TPH Inheritance Problem was that for the fixup for _IncludeAsync method would sometimes return default(Task) (i.e. null) if there was nothing to inclu... — committed to dotnet/efcore by maumar 7 years ago
Adding opt-out mechanism for fix to #9038 - NullReferenceException when using TPH Inheritance — committed to dotnet/efcore by maumar 7 years ago
Adding opt-out mechanism for fix to #9038 - NullReferenceException when using TPH Inheritance — committed to dotnet/efcore by maumar 7 years ago
Improving fix for #9038 - NullReferenceException when using TPH Inheritance Instead of modifying the Task that we return from the problematic query, we are checking if the resulting fixup Task is nul... — committed to dotnet/efcore by maumar 7 years ago
Improving fix for #9038 - NullReferenceException when using TPH Inheritance Instead of modifying the Task that we return from the problematic query, we are checking if the resulting fixup Task is nul... — committed to dotnet/efcore by maumar 7 years ago
[2.0.1] Update the fix for #9038 to make it more robust against multi-collection includes When there are multiple collections being included then we have single fixup task with AwaitMany function whi... — committed to dotnet/efcore by smitpatel 7 years ago
Remove async usage in ListImplementation due to https://github.com/aspnet/EntityFrameworkCore/issues/9038 — committed to IntelliTect/Coalesce by ascott18 7 years ago
Disable async in FindItemAsync due to https://github.com/aspnet/EntityFrameworkCore/issues/9038. Fix dependency detection for select2ajaxMultiple. — committed to IntelliTect/Coalesce by ascott18 7 years ago
update EFCore fix possible NullReferenceException details links https://github.com/aspnet/EntityFrameworkCore/issues/9038 https://github.com/aspnet/EntityFrameworkCore/issues/11153 — committed to WeihanLi/WeihanLi.EntityFramework by WeihanLi 5 years ago

Most upvoted comments

This is something that we’ve immediately noticed after updating. A lot of our queries broke, since we’re using async everywhere. Kinda frustrating that this hasn’t been fixed before stable release.

DSilence on Aug 15, 2017

Any chance for a this (and other 2.0.1 fixes) as MyGet package until there’s a final version?

+1 Would help enormously to get a prerelease.

grokky1 on Sep 4, 2017

Any chance for a this (and other 2.0.1 fixes) as MyGet package until there’s a final version?

TsengSR on Aug 28, 2017

This happens for two-level navigation include, when the first level is empty optional navigation (i.e. the included value is null). We then try to access the null value to include the collection and hence the exception. We are probably missing a null safeguard somewhere - investigating further.

Workarounds:

use sync:

ctx.People.OfType<PersonTeacher>()
	.Include(m => m.Students)
	.ThenInclude(m => m.Family)
	.ThenInclude(m => m.Members)
	.ToList()

break query into multiple includes:

var teachersTask = ctx.People.OfType<PersonTeacher>()
	.Include(m => m.Students)
	.ThenInclude(m => m.Family).ToListAsync();

var familiesTask = ctx.Families.Include(f => f.Members).ToArrayAsync();

teachersTask.Wait();
familiesTask.Wait();

var teachers = teachersTask.Result;

filter out elements that contain null navigation:

var kidsTask1 = ctx.People.OfType<PersonKid>()
	.Where(k => k.Family != null)
	.Include(m => m.Family)
	.ThenInclude(m => m.Members)
	.ToListAsync();

var kidsTask2 = ctx.People.OfType<PersonKid>()
	.Where(k => k.Family == null)
	.ToListAsync();

kidsTask1.Wait();
kidsTask2.Wait();

var kids = kidsTask1.Result.Concat(kidsTask2.Result);

maumar on Jul 11, 2017

Hi, we have a public test feed that you can use to try out the ASP.NET/EF Core 2.0.3 patch!

To try out the pre-release patch, please refer to the following guide:

https://github.com/aspnet/Announcements/issues/274 (info)
https://github.com/aspnet/Home/issues/2254 (discussions and questions)

We are looking for feedback on this patch. We’d like to know if you have any issues with this patch by updating your apps and libraries to the latest packages and seeing if it fixes the issues you’ve had, or if it introduces any new issues. If you have any issues or questions, please reply on this issue to let us know as soon as possible.

Thanks, Eilon

Eilon on Oct 23, 2017

@AndriySvyryd is there any update on a timeline for the 2.0.1 NuGet package to be released? This is blocking my client from migrating to EF Core 2.0. I’m trying to decide if I need to go through and apply your workaround to all my queries, or if I can just wait for 2.0.1 to be released.

chmarti on Oct 15, 2017

Hi guys. Any ETA for this? Is blocking our company update to NET Core 2. Thanks!

marianosz on Aug 22, 2017

@ajcvickers this will create bugs for anyone using TPH Inheritance with ThenInclude and retrieving via Async methods when upgrading to 2.0.0. Should it be higher priority?

caleblloyd on Jul 10, 2017

BTW that feed is not the correct feed, so please be careful using it because it’s a fairly random set of packages on there (many of which are not meant for public consumption). We should have a publicly-consumable feed later this week. I’ll update all “patch” bug issues when that feed is ready.

Eilon on Oct 16, 2017

@danielpmo1371 We’ll ship 2.0.1 as a regular NuGet package, see https://docs.microsoft.com/en-us/nuget/consume-packages/reinstalling-and-updating-packages

AndriySvyryd on Oct 6, 2017

And we will ship 2.0.1 as soon as some technical issues are resolved.

AndriySvyryd on Oct 5, 2017

@danielpmo1371 - This issue has been fixed in 2.0.1 patch.

smitpatel on Oct 5, 2017