kue: (node) warning: possible EventEmitter memory leak detected. 11 job ttl exceeded ack listeners added

I’m running into the same warning when the total amount of jobs (their concurrency value actually) exceeds 10. It doesn’t seem to be anything bad, only a warning, but it’s quite annoying seeing this poping up.

@Shadowys The warning isn’t an issue in this use case. You’re expecting over ten event listeners, so it’s not a memory leak.

You can do some prototype hacks to avoid the error:

// 0 = infinite listeners
require('events').EventEmitter.prototype._maxListeners = 0;

I’m seeing the same issue when I have 10+ subscriptions to different job types…

Does this mean that one should have a separate queue instead of different job types?

I am also facing the same problem mentioned in first comment.

The problem seems to exist only when i write the second process method

queueJobs.process(‘email_job’, 10, function (job, done) { //process the job here

});

queueJobs.process(‘notification_job’, 10, function (job, done) { //process the job here

});

Is it a good practice to only have one process method???

@behrad I faced the issue mentioned in the first comment and upon debugging, it appears to be a case where worker-specific event listening logic is attached to a global queue-level event namespace

https://github.com/Automattic/kue/blob/master/lib/queue/worker.js#L67

  if (self.ttlExceededCb)
    self.queue.removeListener('job ttl exceeded', self.ttlExceededCb);

  self.ttlExceededCb = function(id) {
    if( self.job && self.job.id && self.job.id === id ) {
      self.failed( self.job, { error: true, message: 'TTL exceeded' }, fn );
      events.emit(id, 'ttl exceeded ack');
    }
  }

  /*
   listen if current job ttl received,
   so that this worker can fail current stuck job and continue,
   in case user's process callback is stuck and done is not called in time
   */
  this.queue.on( 'job ttl exceeded', self.ttlExceededCb);

even though there is a check to ensure only one copy of event listener is added, the event-listener itself is worker-scope, so the number of event listeners added to queue’s job ttl exceeded event will be equal to the number of workers, which can easily go above 11. Is there some way by which whatever ttlExceededCb does on a per-worker basis can be done with just one event listener?

I am seeing the same warning as well when I set the concurrency to 20:

queue.process('email', 20, function (job, done) {
    console.log('Processing job: ', job);
    WorkerService.process(job.data, done);
  });

Kue version: 0.11.6

@yangkun2197 queue.setMaxListeners(concurrency + 1)

We are experiencing a similiar issue where jobs are enqueued into two different queues (e.g. eventQueue and notificationQueue). We are using the delay when inserting a job into a queue if that helps diagnose this.

Let me know if you want me to provide the log output for this and I’ll do so ASAP. This is a critical error to look into as its occurring on our production servers which is processing millions of events and notifications daily.

Hello Folks,

We have faced the same issue with the maxListeners of EventEmitter.

The workaround we found was to setMaxListeners at instance level, using queue.setMaxListeners(number).

Do you guys have any suggestion?

Its the sum of the currency limit set on each process. I can’t set this value. So far I have three queue processes, with 8, 5 , 5 limits respectively.

We are also experiencing similar issue. I don’t know what caused this. I have tried using the most minimal setup, but the warning still showed up.

The only thing I can find is, the warning show up when using kue.process method. I have test it by delaying the code that’s calling process method, I also tried commenting it out and no warning.

Whereas on my old application, with an exact setup, everything run smoothly without warning.

Sorry can’t be much of help. Just want to let people know about this error.