OrchardCore: OrchardCore.Menu.Migrations.UpdateFrom3Async causes extreme memory usage and application hanging
I am attempting to update an Orchard Core solution from RC2 to 1.2.2, but UpdateFrom3Async in OrchardCore.Menu.Migrations causes extreme memory usage and does not seem to terminate when we attempt to run the migration on a copy of our production database. For context, the website we are upgrading hosts the documentation for our enterprise software, so we have a large number of menu items to support organization and easy navigation of the help content. To give an idea of scale, we have 43 menus, the largst of which have a few hundred sub-topics, each represented by a menu item. The issue with the migration appears to be that it is attempting to migrate ContentItem.Name to DisplayText for every version of every menu, and we have over 3600 versions of those 43 menus. I added a breakpoint in Visual Studio to log the content item Id of each item being migrated, then started the migration and let it run overnight. It immediately jumped up to ~18GB memory usage, and when I came back to check ~15 hours later, it was still running and had logged ~1.25 million content item IDs, most of which were repeated around 450 times.
I copied the update code in that migration method to one of the projects in our solution for easier debugging, and when I tried changing the predicate in this query
var menus = await _session.Query<ContentItem, ContentItemIndex>(x => x.ContentType == "Menu").ListAsync();
to also filter by only the latest or published version, I got 43 results instead of 3600+, and the migration completed in seconds.
I could imagine that the lack of filtering by latest or published version would be intentional in order to catch and update older versions of content, as a content editor might unpublish a version and revert to an older one, but this is causing major problems in our attempt to update. Is there a more efficient way to accomplish the intended migration here? We have around 23,000 content item versions total across all content types, but because of the way our data is structured, we end up updating the same versions of content items hundreds of times.
About this issue
- Original URL
- State: open
- Created 2 years ago
- Comments: 17 (13 by maintainers)
Thanks, I’ll play around with that. Given our use case, it may be good enough for us to update just the latest and published versions of the menus, which could be accomplished pretty easily by just skipping the migration as you suggested and copying it into one of our modules with a small modification to the query predicate.
And agreed. I think our scenario is a bit out of the ordinary (mostly in having this much data on an RC), but does seem odd that it’s using that much memory. Memory usage jumps up to about 12 GB when debugging from Visual Studio just from executing the initial query before jumping up about 50% once it’s gotten into the loop for a while, though some percentage of that is probably debug info.