efcore: InvalidOperationException : When called from 'VisitLambda', rewriting a node of type 'System.Linq.Expressions.ParameterExpression' must return a non-null value of the same type.

Probably related to https://github.com/aspnet/EntityFrameworkCore/issues/16597 but I am opening because it is marked as fixed for 3.0.0

When performing complex operations, sometimes .Any(x) throws an InvalidOperationException with this message :

An unhandled exception of type ‘System.InvalidOperationException’ occurred in System.Linq.Expressions.dll When called from ‘VisitLambda’, rewriting a node of type ‘System.Linq.Expressions.ParameterExpression’ must return a non-null value of the same type. Alternatively, override ‘VisitLambda’ and change it to not visit children of this type.

Here is a repro. The actual query is meaningless (it does things in a very convoluted way) but it is a recreation of a problem I actually got in a real project, albeit with much more complex objects.

Steps to reproduce

using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Logging.Debug;
using System.Collections.Generic;
using System.Linq;

namespace _18179Repro
{
    class Program
    {
        static void Main(string[] args)
        {
            using var context = new MyContext();

            context.Database.EnsureDeleted();
            context.Database.EnsureCreated();

            context.Customer.Add(new Customer() { });
            context.SaveChanges();

            context.Invoice.Add(new Invoice { CustomerFk = 1 });
            context.SaveChanges();

            context.InvoiceLine.Add(new InvoiceLine { InvoiceFk = 1 });
            context.SaveChanges();

            var invoice = context.Invoice
                .Include(i => i.InvoiceLine)
                .First();

            var customers = context.Customer
                .Select(c => new
                {
                    c.Id,
                    HasInvoiceLines = invoice.InvoiceLine.Any(il => il.InvoiceFk == 1)
                })
                .ToList();
        }
    }

    public partial class MyContext : DbContext
    {
        public virtual DbSet<Customer> Customer { get; set; }
        public virtual DbSet<Invoice> Invoice { get; set; }
        public virtual DbSet<InvoiceLine> InvoiceLine { get; set; }

        private static readonly LoggerFactory Logger = new LoggerFactory(new[] { new DebugLoggerProvider() });

        protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        {
            string connectionString = "Server=.;Database=Repro18179;Trusted_Connection=True;MultipleActiveResultSets=true";

            optionsBuilder.UseSqlServer(connectionString)
                .EnableSensitiveDataLogging()
                .UseLoggerFactory(Logger);
        }

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            modelBuilder.Entity<Customer>(entity =>
            {
                entity.Property(e => e.Id).HasColumnName("id");
            });

            modelBuilder.Entity<Invoice>(entity =>
            {
                entity.HasIndex(e => e.CustomerFk);

                entity.Property(e => e.Id).HasColumnName("id");

                entity.Property(e => e.CustomerFk).HasColumnName("customer_fk");

                entity.HasOne(d => d.CustomerFkNavigation)
                    .WithMany(p => p.Invoice)
                    .HasForeignKey(d => d.CustomerFk)
                    .OnDelete(DeleteBehavior.ClientSetNull)
                    .HasConstraintName("FK_Invoice_Customer");
            });

            modelBuilder.Entity<InvoiceLine>(entity =>
            {
                entity.Property(e => e.Id).HasColumnName("id");

                entity.Property(e => e.InvoiceFk).HasColumnName("invoice_fk");

                entity.HasOne(d => d.InvoiceFkNavigation)
                    .WithMany(p => p.InvoiceLine)
                    .HasForeignKey(d => d.InvoiceFk)
                    .OnDelete(DeleteBehavior.ClientSetNull)
                    .HasConstraintName("FK_InvoiceLine_Invoice");
            });
        }
    }

    public partial class Customer
    {
        public Customer()
        {
            Invoice = new HashSet<Invoice>();
        }

        public int Id { get; set; }

        public virtual ICollection<Invoice> Invoice { get; set; }
    }

    public partial class Invoice
    {
        public Invoice()
        {
            InvoiceLine = new HashSet<InvoiceLine>();
        }

        public int Id { get; set; }
        public int CustomerFk { get; set; }

        public virtual Customer CustomerFkNavigation { get; set; }
        public virtual ICollection<InvoiceLine> InvoiceLine { get; set; }
    }

    public partial class InvoiceLine
    {
        public int Id { get; set; }
        public int InvoiceFk { get; set; }

        public virtual Invoice InvoiceFkNavigation { get; set; }
    }
}

Workaround

Working around this is fortunately trivial : move any query on preloaded objects outside of the main query, like this :

            bool hasInvoiceLines = invoice.InvoiceLine.Any(il => il.InvoiceFk == 1);

            var customers = context.Customer
                .Select(c => new
                {
                    c.Id,
                    HasInvoiceLines = hasInvoiceLines
                })
                .ToList();

Feel free to rename that issue with a more meaningful title if necessary.

Further technical details

EF Core version: 3.0.0 Database provider: Microsoft.EntityFrameworkCore.SqlServer Target framework: .NET Core 3.0 Operating system: Windows 10 x64 IDE: Visual Studio 2019 16.3.1

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Reactions: 23
  • Comments: 24 (6 by maintainers)

Commits related to this issue

Most upvoted comments

I would hope a proposal to “punt” to .NET 5.0 will be refused, given that this issue occurs in .NET Core 3.1 (EFCore 3.1.6 as of writing), which has a Long Term Support policy.

Unfortunately I have learned that LTS in EFCore’s case actually means “if you really want that one fix you need to upgrade to the next major version that has another 50 critical bugs that we will also never fix before the following major revision 2 years from now”. That is true from every single version from 1.0 to now.

Is there any update on when this will be fixed?

This is a major issue for us. It’s affecting too many places in our app to be able to work around all of them.

We are stuck on .Net Core 2.2 and cannot upgrade because of this issue. With 2.2 not being supported at all now, MS has also removed the tags from docker hub for the .Net Core 2.2 container images that we use to create our builds. The 2.2 docker tag references are still working at the moment so we are still able to build, but it feels like a ticking time bomb. If those images are removed from the docker hub (or from wherever they are currently being cached), we are completely screwed!

@jcachat We will look into whether we can patch this.

/cc @smitpatel

I also get the same exception when using Automapper.ProjectTo to map Entity to DTO.

public class ListProductUnitQuantityMapsDto
    {
        public ListProductUnitQuantityMapsDto(int id,
            int companyId,
            string name,
            string description,
            int productId,
            int unitId,
            decimal minUnitQuantity,
            decimal maxUnitQuantity,
            decimal defaultUnitPrice,
            decimal defaultDiscountPercentage,
            decimal defaultDiscountPrice,
            bool isActive,
            bool isDelete,
            DateTime createdAt,
            string createdBy,
            DateTime? lastUpdatedAt,
            string lastUpdatedBy//,
            //string company,
            //string product,
            //string unit
            )
        {
            Id = id;
            CompanyId = companyId;
            Name = name;
            Description = description;
            ProductId = productId;
            UnitId = unitId;
            MinUnitQuantity = minUnitQuantity;
            MaxUnitQuantity = maxUnitQuantity;
            DefaultUnitPrice = defaultUnitPrice;
            DefaultDiscountPercentage = defaultDiscountPercentage;
            DefaultDiscountPrice = defaultDiscountPrice;
            IsActive = isActive;
            IsDelete = isDelete;
            CreatedAt = createdAt;
            CreatedBy = createdBy;
            LastUpdatedAt = lastUpdatedAt;
            LastUpdatedBy = lastUpdatedBy;
            //Company = company;
            //Product = product;
            //Unit = unit;
        }

        public int Id { get; private set; }
        public int CompanyId { get; private set; }
        public string Name { get; private set; }
        public string Description { get; private set; }
        public int ProductId { get; private set; }
        public int UnitId { get; private set; }
        public decimal MinUnitQuantity { get; private set; }
        public decimal MaxUnitQuantity { get; private set; }
        public decimal DefaultUnitPrice { get; private set; }
        public decimal DefaultDiscountPercentage { get; private set; }
        public decimal DefaultDiscountPrice { get; private set; }
        public bool IsActive { get; private set; }
        public bool IsDelete { get; private set; }
        public DateTime CreatedAt { get; private set; }
        public string CreatedBy { get; private set; }
        public DateTime? LastUpdatedAt { get; private set; }
        public string LastUpdatedBy { get; private set; }
        public string Company { get; private set; }
        public string Product { get; private set; }
        public string Unit { get; private set; }
    }

Here I’m getting exception.

 public async Task<IList<ListProductUnitQuantityMapsDto>> ListAll()
        {
            return await Mapper.ProjectTo<ListProductUnitQuantityMapsDto>(NoTrackingEntity)
                .ToListAsync();
        }

Note from triage: Since invoiceLines is captured from outside the query, the entire invoiceLines.Any(il => il.InvoiceFk == 1) should be evaluated by the funcletizer.

Resolving this issue will only update the exception message thrown. The query is going to throw exception either way as LambdaExpression cannot be translated to SQL.

The exception message comes while evaluating a lambda expression. Any lambda which are being passed to queryable methods will throw client eval exception. If you have any other kind of lambda then it is client eval and it may not work in all cases.

In very specific case in OP. the lambda expression is being applied on client code and can be fully extracted out in a variable outside of the query. In future release, EF core would do that automatically in parameter extraction. Above scenario cannot be patched because it touches every query path (even cached ones). The work-around is really easy (which EF Core would do under the hood anyway). From query in OP

            var invoice = context.Invoice
                .Include(i => i.InvoiceLine)
                .First();

var hasInvoceLines = invoice.InvoiceLine.Any(il => il.InvoiceFk == 1); //since this is not correlated to customer, better to pre-evaluate this.
            var customers = context.Customer
                .Select(c => new
                {
                    c.Id,
                    HasInvoiceLines = hasInvoceLines
                })
                .ToList();

If your issue is not same as OP then file a new issue with repro code. Then we can separately evaluating about patching.

@fschlaef maybe for now, you can change as below that.


class Program
    {
        static void Main(string[] args)
        {
            using var context = new MyContext();

            context.Database.EnsureDeleted();
            context.Database.EnsureCreated();

            context.Customer.Add(new Customer() { });
            context.SaveChanges();

            context.Invoice.Add(new Invoice { CustomerFk = 1 });
            context.SaveChanges();

            context.InvoiceLine.Add(new InvoiceLine { InvoiceFk = 1 });
            context.SaveChanges();

            IQueryable<InvoiceLine> invoiceLines = context.InvoiceLine;

            var customers = context.Customer
                .Select(c => new
                {
                    c.Id,
                    HasInvoiceLines = invoiceLines.Any(il => il.InvoiceFk == 1)
                })
                .ToList();
        }
    }