framework: Memory leak on database insert function
- Laravel Version: 6.0
- PHP Version: 7.2.14
- Database Driver & Version: PDO, 5.7.21
Description:
I was trying to transfer 100 000 records from one table to another and got some kind of memory leak while inserting records. I checked same script on laravel 5.8 version and worked great. It seems on the new version something is not cleared from memory after inserting
Steps To Reproduce:
\DB::connection()->disableQueryLog();
DB::table('test_table2')->orderBy('id')->chunk(500, function ($chunk) {
$data = [];
foreach ($chunk as $record) {
$data []= [
'test' => $record->test
];
}
DB::table('test_table')->insert($data);
dump(memory_get_usage());
});
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Reactions: 3
- Comments: 15 (6 by maintainers)
I don’t know if this solves the above issues, but I had this issue too and found a cause.
In the
logQuery()function, commenting out 687 got rid of my memory leak. The other lines had no effect. Ergo: someone was saving queries in an Event Listener. https://github.com/laravel/framework/blob/ea3d291420ffd5ebcc6bb7c1deef32c853ce24a8/src/Illuminate/Database/Connection.php#L685-L692This someone turned out to be Flare/Ignition, which ships with Laravel since 6.0.
To disable their logging, publish the config by running:
And then under
flare.reporting, changereport_queries,maximum_number_of_collected_queriesand/orreport_query_bindings.To disable events only in current connection you can use:
DB::getConnection()->unsetEventDispatcher();For me, the problem was a telescope database logging. It logs all queries even with
\DB::connection()->disableQueryLog().I added my artisan command to
telescope.ignoreCommandsconfig and problem solved. So be aware when using any logging packages. They can cause the mess instead of a framework.This is very much still an issue. The default Laravel install will include “spatie/laravel-ignition” with an unpublished config file that collects database queries (see
flare_middlewareconfig key) which will behave as a memory leak.Confirming the issue with 5.5 (Symfony + EloquentBundle + illuminate/database-5.5).
And memory leaks with every query more and more.
$connection->logging()returns false;Doing the same with bare
$connection->getPdo()works fine;