framework: Memory leak on database insert function

  • Laravel Version: 6.0
  • PHP Version: 7.2.14
  • Database Driver & Version: PDO, 5.7.21

Description:

I was trying to transfer 100 000 records from one table to another and got some kind of memory leak while inserting records. I checked same script on laravel 5.8 version and worked great. It seems on the new version something is not cleared from memory after inserting

Steps To Reproduce:

     \DB::connection()->disableQueryLog();
     DB::table('test_table2')->orderBy('id')->chunk(500, function ($chunk) {
            $data = [];
            foreach ($chunk as $record) {
                $data []= [
                    'test' => $record->test
                ];
            }
            DB::table('test_table')->insert($data);

            dump(memory_get_usage());
        });

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Reactions: 3
  • Comments: 15 (6 by maintainers)

Most upvoted comments

I don’t know if this solves the above issues, but I had this issue too and found a cause.

In the logQuery() function, commenting out 687 got rid of my memory leak. The other lines had no effect. Ergo: someone was saving queries in an Event Listener. https://github.com/laravel/framework/blob/ea3d291420ffd5ebcc6bb7c1deef32c853ce24a8/src/Illuminate/Database/Connection.php#L685-L692

This someone turned out to be Flare/Ignition, which ships with Laravel since 6.0.

To disable their logging, publish the config by running:

php artisan vendor:publish --provider="Facade\Ignition\IgnitionServiceProvider" --tag="flare-config"

And then under flare.reporting, change report_queries, maximum_number_of_collected_queries and/or report_query_bindings.

I don’t know if this solves the above issues, but I had this issue too and found a cause.

In the logQuery() function, commenting out 687 got rid of my memory leak. The other lines had no effect. Ergo: someone was saving queries in an Event Listener. https://github.com/laravel/framework/blob/ea3d291420ffd5ebcc6bb7c1deef32c853ce24a8/src/Illuminate/Database/Connection.php#L685-L692

This someone turned out to be Flare/Ignition, which ships with Laravel since 6.0.

To disable their logging, publish the config by running:

php artisan vendor:publish --provider="Facade\Ignition\IgnitionServiceProvider" --tag="flare-config"

And then under flare.reporting, change report_queries, maximum_number_of_collected_queries and/or report_query_bindings.

To disable events only in current connection you can use:
DB::getConnection()->unsetEventDispatcher();

For me, the problem was a telescope database logging. It logs all queries even with \DB::connection()->disableQueryLog().

I added my artisan command to telescope.ignoreCommands config and problem solved. So be aware when using any logging packages. They can cause the mess instead of a framework.

Closing this issue because it’s inactive, already solved, old or not relevant anymore. Feel free to reply if you’re still experiencing this issue and we’ll re-open this issue.

This is very much still an issue. The default Laravel install will include “spatie/laravel-ignition” with an unpublished config file that collects database queries (see flare_middleware config key) which will behave as a memory leak.

Confirming the issue with 5.5 (Symfony + EloquentBundle + illuminate/database-5.5).

$connection->disableQueryLog();
$connection->table($tableName)->insert($array);

And memory leaks with every query more and more. $connection->logging() returns false;

Doing the same with bare $connection->getPdo() works fine;