efcore: Simple query filter breaks simple projection

As reported by @klaussj here: https://github.com/aspnet/EntityFrameworkCore/issues/12951#issuecomment-427380358

Removing the query filter fixes the issue.

namespace DemoIssue
{
    public class Entity
    {
        public int Id { get; set; }
        public int? RefEntityId { get; set; }
        public RefEntity RefEntity { get; set; }
    }

    public class RefEntity
    {
        public int Id { get; set; }
        public bool Public { get; set; }
    }

    public class EntityDto
    {
        public int Id { get; set; }
        public int? RefEntityId { get; set; }
        public RefEntityDto RefEntity { get; set; }
    }

    public class RefEntityDto
    {
        public int Id { get; set; }
        public bool Public { get; set; }
    }

    public class DbTestContext : DbContext
    {
        public DbSet<Entity> Entities { get; set; }
        public DbSet<RefEntity> RefEntities { get; set; }

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            modelBuilder.Entity<RefEntity>().HasQueryFilter(f => f.Public == true);
        }

        public DbTestContext(DbContextOptions<DbTestContext> options) : base(options)
        {

        }
    }

    class Program
    {
        static void Main(string[] args)
        {
            using (var connection = new SqliteConnection("DataSource=:memory:"))
            {
                connection.Open();

                var options = new DbContextOptionsBuilder<DbTestContext>()
                    .UseSqlite(connection)
                    .Options;

                using (var db = new DbTestContext(options))
                {
                    db.Database.EnsureCreated();

                    db.RefEntities.Add(new RefEntity()
                    {
                        Id = 1,
                        Public = false
                    });

                    db.Entities.Add(new Entity()
                    {
                        Id = 1,
                        RefEntityId = 1
                    });

                    db.SaveChanges();

                    var notWorking = db.Entities.Select<Entity, EntityDto>(s =>
                        new EntityDto
                        {
                            Id = s.Id,
                            RefEntity = s.RefEntity == null ?
                                null :
                                new RefEntityDto()
                                {
                                    Id = s.RefEntity.Id,
                                    Public = s.RefEntity.Public
                                },
                            RefEntityId = s.RefEntityId
                        }).Single(p => p.Id == 1);
                }
            }
        }
    }
}

Exception:

Unhandled Exception: System.InvalidOperationException: Nullable object must have a value.
   at lambda_method(Closure , QueryContext , TransparentIdentifier`2 )
   at Microsoft.EntityFrameworkCore.Query.ExpressionVisitors.Internal.ProjectionShaper.TypedProjectionShaper`3.Shape(QueryContext queryContext, ValueBuffer& valueBuffer)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable`1.Enumerator.BufferlessMoveNext(DbContext _, Boolean buffer)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable`1.Enumerator.MoveNext()
   at System.Linq.Enumerable.Single[TSource](IEnumerable`1 source)
   at Microsoft.EntityFrameworkCore.Query.Internal.LinqOperatorProvider.ResultEnumerable`1.GetEnumerator()
   at Microsoft.EntityFrameworkCore.Query.Internal.LinqOperatorProvider.ExceptionInterceptor`1.EnumeratorExceptionInterceptor.MoveNext()
   at System.Linq.Enumerable.TryGetFirst[TSource](IEnumerable`1 source, Boolean& found)
   at System.Linq.Enumerable.First[TSource](IEnumerable`1 source)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass15_1`1.<CompileQueryCore>b__0(QueryContext qc)
   at System.Linq.Queryable.Single[TSource](IQueryable`1 source, Expression`1 predicate)
   at DemoIssue.Program.Main(String[] args) in C:\Stuff\TwoOneCore\TwoOneCore\Program.cs:line 79

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 24 (12 by maintainers)

Commits related to this issue

Most upvoted comments

What a triage. Instead of better documentation - did it ocur to you to issue a hotfix that fixes a bug that makes a whole subsystem (i.e. query filters) totally unusable? And it is nice to see that you basically tell all of us to go home and use EF - because a bug that shuts down a whole subsystem is scheduled for - ah, 3.0. Nice. Basically “oh, sorry you rely on query filterrs, maybe in a year we prouce a usable product there”.

This is a 1.0 feature that does not work. 2.1.5 is the next patch. It should get this into a usable state. At minimum 2.2 should include an emergency fix for this. Otherwise basically EF is it - I do not care about EF Core because now you actually tell me that the new features that make it worthwhile and everyone loves pointing out - do not work and will not get fixed.

Any manager going through tirage and handing out reprimands? Bevcause this is the 2nd time this particular bug is either not accepted as bug (hey, fix your data) or - ah, scheudled for some far point in the future, usage and impact on users being irrelevant. I personally can only be happy that I am slowly removing query filters anyway (related to some business logic that makes them jsut the wrong place - i.e. we will filter on the REST side, not the model, as some related objects may be “oiutdated” but in our case then still are visible read only, so general filters do not work). But I can bet other people actually use that and are not happy about a 3.0 schedule.

@ajcvickers @smitpatel @maumar I’m working on migrating a very large Fintech app from Net Framework to Net Core and this now completely blocking our company moving forward. Is there any chance you could produce a small fork with the work around or tell me where to remove the optimisation myself so that we can push on? What’s the justification for leaving this until version 3, it seems to be pretty core functionality?

Many thanks for in advance.

The code I highlighted only addresses the case where EF is expanding navigation into a JOIN. If you create the joins yourself, and the query involves client evaluation you need to add your own null protection. You can do it using : ? operator, like so:

from c in ctx.Customers
join o in ctx.Orders on c.Name equal o.CustomerName into grouping
from o in grouping.DefaultIfEmpty()
select o != null ? (int?)o.InvoiceNumber : null

If the query is fully translated, the ? : part will be optimized out during the translation, so the SQL should look exactly the same with and without this part. However, if client evaluation is present it should prevent the error from happening.

What you might also be seeing is that one of the result properties is expected to be non-nullable, but it ends up as null.

You will get this error for queries like:

from c in ctx.Customers
join o in ctx.Orders on c.Name equal o.CustomerName into grouping
from o in grouping.DefaultIfEmpty()
select o.InvoiceNumber

result is expected to be of type int, but SQL returns a null value and we cant fit it into the expected result type.