framework: Memory leaks in PHPUnit

Easy way for you to handle this would extending directly from PHPUnit and letting the framework aside. Hence, you will indeed remove these issues since DB is not in place anymore.

Now, if you wanna have DB interaction just for the peace of mind, I would suggest moving them as feature. Once they are there, create two test suits. One for Unit and One for feature and start moving tests from feature to unit.

This is not an answer for your issue, but I would start from there since I had to do the same in the past once you hit a big number of tests.

I noticed that my memory usage increased after upgrading to PHP 7.4.0 - suddenly started hitting the default 128 MB limit. I did the same thing with memory_get_usage() but put it down to being an issue on my end, so glad to see it’s not just me.

For reference: Laravel 6.6.1, PHP 7.4.0, Postgres 12.1.

I just tested against PHPUnit 7 as well to compare - similar memory creep but to a lesser extent.

PHPUnit 7.5.17: 137.00 MB PHPUnit 8.4.3: 165.00 MB

@brendt ,

I was able to find a workaround for this within a L7 / PHP 7.4 application. It didn’t entirely fix the issue, but it did drastically reduce it. Our two main issues were coming from our route files and database factories. We started caching routes prior to tests, and modified the test suite to keep the base factory builder around between tests.

It’s worth mentioning that class-based database factories (available in L8) should see a dramatic memory usage drop, as the registration and such under the hood is done is such a way that doesn’t trip the memory leak in PHP.

Here’s how we kept the database factory around between tests:

<?php

namespace Tests\Concerns;

use Illuminate\Contracts\Console\Kernel;
use Illuminate\Database\Eloquent\Factory as EloquentFactory;

trait CreatesApplication
{
    /**
     * The statically cached eloquent factory implementation.
     *
     * @var \Illuminate\Database\Eloquent\Factory
     */
    protected static $factory = null;

    /**
     * Creates the application.
     *
     * @return \Illuminate\Foundation\Application
     */
    public function createApplication()
    {
        $app = require __DIR__.'/../../bootstrap/app.php';

        $app->make(Kernel::class)->bootstrap();

        if (! is_null(static::$factory)) {
            $app->instance(EloquentFactory::class, static::$factory);
        }

        $this->beforeApplicationDestroyed(function () use ($app) {
            if (! is_null(static::$factory)) {
                return;
            }

            if ($app->bound(EloquentFactory::class)) {
                static::$factory = $app->make(EloquentFactory::class);
            }
        });

        return $app;
    }
}

This code waits for a test to need any model factory, and then once the base factory manager is bound to the application, it’s reinjected so that it doesn’t have to rebuild again.

Prior to these changes, we had maybe 50 MB leaking per 60 tests. After these changes, it dropped to ~1 MB leaking per 60 tests. Feature tests leaked a little more due to views, but wasn’t more than ~5 MB per 60 tests.

It’s not perfect, but it’s much better than what we started with. It’s given my team enough breathing room until we have the capacity to upgrade to PHP 8.1.

For anyone who comes here after, I found an old bug report in the php framework that explained the root cause to be cycles and armed with this info I added a teardown to my base test class that catches these. https://bugs.php.net/bug.php?id=75914 protected function tearDown(): void { parent::tearDown(); gc_collect_cycles(); }

I remember seeing this and other problems when writing my first tests, in a L5.1 application. Until I changed in phpunit.xml the option processIsolation to true. It’s slower and patchy this way, but the memory usage kept from growing.

Can we maybe create a simple test case which can reproduce this?

Here we go: https://github.com/mfn/laravel-tests-memleak

Upon checkout:

• composer install
• php artisan generateHorde 50
• vendor/bin/phpunit Observe ever increasing memory

A bare bones installation with only adjustments for creating senseless tests. No database involved.

I just remembered some other issue/PR which talked about memory leaks where the culprit was some, I think, closure/reference related leak (but AFAIK this had to do with the Query builder or Eloquent, which doesn’t apply in this case).

Or maybe it’s such kind of issue with how the application is booted for the tests.

We found an issue in https://github.com/php/php-src/pull/5581 which caused a memory leak in PHP 8.1 in case destructors are present. Maybe it’s related. A fix is ready in https://github.com/php/php-src/pull/9265

Is this issue similar to this one with Symfony: https://jolicode.com/blog/you-may-have-memory-leaking-from-php-7-and-symfony-tests? It is related to this issue in PHP: https://bugs.php.net/bug.php?id=76982. They mitigated it in Symfony, not sure if something similar can be used in Laravel.

I’m looking at this again and I’ve found the Symfony issue and workaround they implemented for them:

Issue: https://github.com/symfony/symfony/issues/32220 Workaround: https://github.com/symfony/symfony/pull/32236

Afaik, Symfony already released these and Laravel should run with these, including @mfn’s test repo: https://github.com/mfn/laravel-tests-memleak

Does anyone has an idea how we could potentially work around this ourselves in Laravel? Otherwise I’m afraid there’s not much we could do.

@brendt shouldn’t we try to bring this to the attention of @sebastianbergmann and @robertbasic? I don’t think there’s much we can do ourselves here?

It’s still broken with mockery 1.3.1

This is the memory usage of my testsuite, running 1142 tests