knex: Memory leak in 0.14.x

Environment

Knex version: 0.14.2 Database + version: Postgres 9.4 OS: Debian

Bug

I’ve been experiencing quite the extensive memory leak since having updated my production app’s knex version from 0.13 to 0.14. Leaving the app running from friday afternoon 'til monday morning resulted in ~500mb rss which never decreased after that.

After a lot of struggling and testing I managed to eventually get a pair of heapdumps that when compared still didn’t make much sense but they did contain a lot of array and closure references to the underlying generic-pool module. This led me to believe the source was from knex, which also adds up to the changelog where generic pool was upgraded.

The pool options are as follows:

connectionOptions.pool = {
	max: dbCfg.poolMax,
	min: 0,
	idleTimeoutMillis: 5000,
	evictionRunIntervalMillis: 1000,
	Promise,
};

Due to the nature of this leak being ‘very slow but extensive’, it is quite hard to reproduce as it takes a while for the memory to build up, and most likely relies on heavy traffic. For this reason I cannot guarantee that the leak is in generic-pool or if it is in knex, hence why this issue is not reported directly into the generic-pool repo.

After rolling back to v 0.13 of knex and leaving production app running for an equal amount of days this week, the app is still steady and fine at ~120mb, despite a lot more traffic during weekdays compared to weekends.

Has anyone else been experiencing something similar since upgrading to 0.14.2?

Unfortunately I’m pretty backed up with work until few weeks after new years, so I can’t dig too much into this at this time. But I felt it neccessary to have this information available for others.

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Comments: 16 (15 by maintainers)

Most upvoted comments

this is actually a node-postgres bug it looks like brianc/node-postgres#1417

I just release straight 0.14.3 later on this week.

@elhigu Great news! Any chance for an early release after #2450 lands, e. g. in RC form?

I’ll try to write test for this one.

@tgriesser +1000 for defaulting to client default when possible, that’s something I would like to see happen.

I would happily take the time to implement this for pg as a starting point if you’re on board.