mongoose: eachAsync with async function causes memory leak

Do you want to request a feature or report a bug? Bug

What is the current behavior? When iterating on a query cursor with eachAsync and an async function as callback the memory quickly grows until the node app crashes.

If the current behavior is a bug, please provide the steps to reproduce. No memory leak:

const playerCursor: QueryCursor<IPlayerProfileModel> = PlayerProfile.find({}, projection).lean().cursor();
    await playerCursor.eachAsync(
      (profile: IPlayerProfileModel) => {
        return;
      },
      { parallel: 50 }
    );

Memory leak:

const playerCursor: QueryCursor<IPlayerProfileModel> = PlayerProfile.find({}, projection).lean().cursor();
    await playerCursor.eachAsync(
      async (profile: IPlayerProfileModel) => {
        return;
      },
      { parallel: 50 }
    );

What is the expected behavior?

Iteration on cursor, with max 50 async functions running concurrently without having a memory leak. The same issue appears with a concurrency of 1 (the memory just grows way slower then).

Please mention your node.js, mongoose and MongoDB version. Node 8.2.1, Mongoose 4.13.5 MongoDb 3.4

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Comments: 15 (2 by maintainers)

Most upvoted comments

Yeah that’s strange, rss, the resident set size, is growing monotonically. Same thing on my machine, that’s strange that the heap is growing and shrinking but the resident set size isn’t. I need to investigate this more.

Re: async, I’m not certain about the perf implications but at the very least, without async we go on to the next doc immediately, whereas with async mongoose waits until at least the next tick of the event loop. I don’t have a good answer for why its that much slower but that fact may have something to do with it.

Definitely shouldn’t leak memory, will investigate ASAP 👍

Confirmed, adding async makes heapUsed grow monotonically. Good catch, thanks for your patience. Will investigate more later today 👍