efcore: Projection with nested projection hangs/does not complete in 2.0 Preview 2

When we updated our code base to 2.0 Preview-2 we had significant problems with a number of our queries hanging. We initially flagged this as AutoMapper/AutoMapper/#2184. But during discussions it has become apparent that this is a change at the EF Core 2 level.

The following style of query with an inner projection now fails to complete / hangs:

var result = await context.Documents.Select(d => new DocumentDTO
    {
        Reference = d.Reference,
        Items = d.Items.Select(i => new DocumentItemDTO
        {
            Reference = i.Reference
        }).ToList()
    }).FirstAsync();

It used to work in 2.0 Preview 1 (and 1.x.x) but now hangs in 2.0 Preview 2.

I appreciate that it is a problematic query, in the sense that the inner projection uses ToList() whilst the outer projection is FirstAsync() and I have some ideas for workarounds

But overall I feel that this should continue to work, it feels natural, it is not unreasonable to request nested data and a projection in one step (we are trying to reduce the data retrieved from the db server) and I can’t see how you could rewrite the inner projection as async.

It seems not unreasonable to ask EF Core to resolve this type of query correctly and return the data requested asynchronously regardless of how inefficient / ugly it is.

I have created the following gists to repro this.

Discussions on the AutoMapper repo have suggested that this is related to #8208 and possibly #9029.

Meanwhile, we’ve gone back to 2.0 Preview 1 and are trying to work out how we are going to address this in 2.0 Final. So any thoughts/fixes etc. would be much appreciated. Many thanks.

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Reactions: 7
  • Comments: 35 (15 by maintainers)

Commits related to this issue

Most upvoted comments

@clement911 I posted on 7/16 that they shouldn’t go RTM with this obvious and very common issue that causes hard deadlocks, the product is useless with this massive gotcha in there, especially if you started building something that worked fine in earlier previews. It’s not an incorrect pattern. Doesn’t seem like whoever is making decisions cares much about quality, only deadlines. Makes me question about using or recommending .NET Core at all if this is the attitude they have.

In my personal opinion, this is quite a major issue and 2.0 shouldn’t be released until async queries with nested projections can be safely written without deadlocking. It worked in preview1, it should not result in a deadlock in an RTM release…

I have isolated the issue and filed #10188

@kodelapan - Use following query as work-around

var result = (from a in DnaCoreContext.Article
                          join c in DnaCoreContext.ArticleCategory on a.CategoryId equals c.Id
                          where status.Contains(a.Status) && c.IsVisible == true
                          orderby a.CreatedDate descending
                          select new Article
                          {
                              Id = a.Id,
                              ShortDescription = a.ShortDescription,
                              Slug = a.Slug,
                              Title = a.Title,
                              CategoryId = a.CategoryId,
                              CreatedById = a.CreatedById,
                              Category = new ArticleCategory
                              {
                                  Id = c.Id,
                                  Name = c.Name
                              },
                              ImageMaps = (from map in a.ImageMaps
                                           join i in DnaCoreContext.Image on map.ImageId equals i.Id
                                           where i.IsPrimary == true
                                           select new ArticleImage
                                           {
                                               Id = map.Id,
                                               Image = new Image
                                               {
                                                   Id = i.Id,
                                                   FileName = i.FileName,
                                                   FileExtension = i.FileExtension,
                                                   ImageUrl = i.ImageUrl
                                               }
                                           }).ToList()
                          });

            if(staffId > 0 )
            {
                result = result.Where(x => x.CreatedById == staffId);
            }
            return new PaginationEntity<Article>
            {
                Page = pageIndex,
                PageSize = pageSize,
                TotalCount = await result.CountAsync(),
                Items = await result.Skip((pageIndex - 1) * pageSize).Take(pageSize).ToListAsync()
            };

(it worked for me). The difference is, instead of using DnaCoreContext.ImageMaps following by where map.ArticleId == a.Id it uses the a.ImageMaps navigation property which will give same result. You can do similar for any other queries you have.

I’m getting the same issue with the EF Core 2.0 (final). Wondering whether any positive workaround is out there yet? Our solution was working well with EF Core 1.1.2. Its really annoying when the final versions have fundamental bugs like this. Really don’t get the point of releasing a product having obvious known issues that almost make it useless for most of the users. This results only waste of time!

Out of interest, I was having this issue and saw some further strangeness: when I add an ‘Include’ for the property of the inner projection, the deadlock goes away. I get a warning logged saying the Include is not necessary because of the projection, and will be ignored, but it certainly changes the behaviour.

It’s not a good workaround for this issue though, because it looks like EF core then does separate subqueries for each item.

var blogs = await _context.Blogs
    .Include(blog => blog.Posts)  // adding this prevents deadlock, but causes multiple subqueries
    .Select(blog => new
    {
        Id = blog.BlogId,
        Url = blog.Url,
        Posts = blog.Posts.Select(p => p.Title).ToArray()
    })
    .ToListAsync();

@anpete Do we need a new issue for patch approval?

Hi,

This is a major concern for us, all our framework is based on async/await, and queries are blocked. We actually consider moving to EF 6, unfortunately it is a show-stopper.

Do you have a time frame?

Thanks in advance

@bridgesquared Thanks for reporting this–we will investigate, but we will not be able to get a fix in for 2.0. Some workarounds to try:

  • Use First instead of FirstAsync. (Since the inner query is not async there will be some blocking anyway.)
  • Remove the ToList call and then enumerate Items explicitly. (This may change the way the DTO gets consumed, so look into your app code as to whether this is acceptable.)
  • Split it explicitly into two queries and then combine the results into the DTO afterwards.