framework: [6.x] Memory leak on Redis queue when using relation

Laravel Version: 6.0.4
PHP Version: 7.3.9
Database Driver & Version: MySQL and Redis with predis

Description:

It seems that the worker is leaking memory. I already search for this issue, it seems to exists on multiple Laravel version (I tested 5.6, 5.8 and 6.x). It seems that the leak is more important when having models with loaded relations in the Job to handle.

Steps To Reproduce:

Create a new laravel project.
Configure a MySQL DB and a Redis connection for queue.
Create a model which will be related to User model (such as PasswordReset).
Add the relation method with PasswordReset to the User model.
Create a Job which implements ShouldQueue, and which take a User as constructor argument. To see a big difference in memory usage, use load on the User to load the relationship with PasswordReset.
Add debug memory echo to the Illuminate\Queue\Worker process method, just after the try, and just before catch (you can echo result of memory_get_usage for example).
Run the queue:work command.
Dispatch job to the queue, you will see memory usage is increasing after each job, without lowing even if the queue is free.

References and why am I posting this issue

I know there have been multiple issue about queue memory leaking. Such as #25999.

They say “PHP is meant to die” and “just use queue:listen”.

But we are using Horizon, and I think we cannot configure the command it is running on child process, am I right ? Because of Horizon with 10 child process, and queue which runs a lot of job, our queue worker dyno is crashing each 12 hours. For the moment, I configured a horizon:terminate with a daemon auto restart each hour, but that’s not a real solution.

Can you help ? 😃

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 29 (25 by maintainers)

Most upvoted comments

There are two memory leaks:

~~one in the Eloquent QueryBuilder preventing it to be garbage collected, I sent a pull request~~
and a small one in Symfony console output, it is ~12 byte/job and does not matter if --quiet is used, I will open an issue there

edit: turns out I just optimized it for faster gc

netpok on Oct 11, 2019

@paul-thebaud I can confirm the leak; I followed your instructions from https://github.com/laravel/framework/issues/30114#issuecomment-536510493

I pushed 1000 jobs each time I did a test.

With your vanilla code:

after first job: 15813904
after ~500 jobs: 17474720
after 1000 jobs: 19344040

When I remove the Eloquent code (no constructor args, not storing anything in the job), I got this:

after first job: 13665120
after ~500 jobs: 13673280
after 1000 jobs: 13677376

So it seems the basic system itself is leaking already memory, but with an Eloquent in the mix it’s amplified.

My takeaway from this is that the leak itself is connected to the job class. The “smallest” job class (doing basically nothing) already leaks, so I wonder if something is holding a reference to it or some other information about it (or its past).

The empty Job class consumed 12kb after 1000 runs: 13677376 - 13665120 = 12256

That would make 12256 / 1000 = 12.256, i.e. ~12 bytes per job?

mfn on Oct 5, 2019

I kinda stopped using horizon on new projects, so for now I’ll try to run the queue without a daemon and see how that works.

THANKS!

amosmos on Jun 6, 2020

I have the same problem running queue with redis. The queue:work command stopped irregularly and it is hard to know where is the troublesome code.

maitanwebikejp on Oct 4, 2019