symfony: [Framework bundle] Cleanup error and exception handlers on Kernel shutdown

Symfony version(s) affected

7.0.0

Description

PHPUnit 11 checks for any leftovers in error handlers https://github.com/sebastianbergmann/phpunit/pull/5619

How to reproduce

Run FrameworkBundle->boot() in phpunit test.

https://github.com/symfony/symfony/blob/815df931dca4663a6375e205b0fdf6e546ebfa75/src/Symfony/Bundle/FrameworkBundle/FrameworkBundle.php#L98

It results in Test code or tested code did not remove its own exception handlers thrown by phpunit.

Possible Solution

Introduce FrameworkBundle::shutdown() that would cleanup error handler set in boot(). IMO it’s reasonable to have some cleanup in shutdown() of things initiated in boot().

Additional Context

No response

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 22
  • Comments: 46 (32 by maintainers)

Commits related to this issue

Most upvoted comments

A different workaround, seeing as the Symfony/Errorhandler is a singleton and will only register once, is to register the error handler before phpunit memorizes all the existing error handlers. This way the error handler Symfony register’s is not new.

Create (or modify) /tests/bootstrap.php:

<?php
declare(strict_types=1);

use Symfony\Component\ErrorHandler\ErrorHandler;

ErrorHandler::register(null, false);

and in phpunit.xml

<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/11.0/phpunit.xsd"
         bootstrap="tests/bootstrap.php">
...
</phpunit>

@frankdekker is right, the error handler registered by FrameworkBundle would not replace the active error handler. PHPUnit already registers its own error handler and that one is still active at the end of the test. So we’re good here.

The problem which PHPUnit reports is about the exception handler. Yes, error handler and exception handler are two different things. And yes, it’s super confusing that Symfony has a method called ErrorHandler::register() that registers an error handler and an exception handler. 😓

PHPUnit on the other hand does not register an exception handler and even if it did, that handler would be wrapped by ErrorHandler::register() which PHPUnit would still rightfully complain about.

But. ErrorHandler won’t replace the exception handler if ErrorHandleralready is the current exception handler. This means, we can simplify @frankdekker’s bootstrap script like this:

<?php

declare(strict_types=1);

use Symfony\Component\ErrorHandler\ErrorHandler;

require dirname(__DIR__).'/vendor/autoload.php';

set_exception_handler([new ErrorHandler(), 'handleException']);

That being said, this feels like a hack. Maybe we should add a way to opt out of the exception handler. In situations where the kernel is executed within a huge try/catch block (which a PHPUnit test basically is), that exception handler does not serve any purpose anyway.

I had the same problem with Symfony 6.4 and PHPUnit 11. I temporarily solved it by calling this method in the tearDown of my base test case (which extends from KernelTestCase or WebTestCase):

protected function restoreExceptionHandler(): void
{
    while (true) {
        $previousHandler = set_exception_handler(static fn() => null);

        restore_exception_handler();

        if ($previousHandler === null) {
            break;
        }

        restore_exception_handler();
    }
}

The tearDown looks like this:

protected function tearDown(): void
{
    parent::tearDown();

    $this->restoreExceptionHandler();
}

It is based on what Laravel did on their testsuite: https://github.com/laravel/framework/pull/49622/files

Maybe the solution is to call a similar method in the tearDown of KernelTestCase and WebTestCase?

Same issue “Test code or tested code did not remove its own exception handlers” here with

  • PHP 8.2.15
  • Symfony 6.4.3
  • phpunit 11

Please, no more me-toos here. We know the issue exists and a workaround has been posted. Let’s focus on solutions.

What if we stopped registering error handlers, exception handlers or shutdown functions of any kind in FrameworkBundle?

We do need this stuff way before the framework is booted, so whatever FrameworkBundle registers here, it does it way too late. This applies especially to exception handlers: They are meant to handle uncaught exceptions. The kernel however runs in a big try/catch, so at the time FrameworkBundle is booted, the need for an exception handler is more or less over.

In modern Symfony applications, the error handling should be set up by the runtime already. All that FrameworkBundle does is connecting the logger properly. The logic that registers an error handler or exception handler is usually dormant and causes trouble because PHPUnit tests run outside of the runtime.

interesting.

this problem is absent on symfony 6.4, but exists in latest 7.2

This issue is a blocker for PHPUnit 11 in any Symfony project that uses KernelTestCase or WebTestCase. Given that PHPUnit comes with its own error handler, maybe the solution would be to opt out of Symfony’s own error handler when booting the kernel for functional tests?

I’m using Laravel 11 and PHP Unit 11 and nothing fixes this issue. I can only run my tests on PHP Unit 10. Is anyone still experiencing this?

Test code or tested code removed error handlers other than its own

Test code or tested code removed exception handlers other than its own

thanks very much @derrabus, you’re very simple addition to my pre-existing tests/bootstrap.php resolved the issue:

use Symfony\Component\ErrorHandler\ErrorHandler;
set_exception_handler([new ErrorHandler(), 'handleException']);

I agree with you that, while simple, it does seem somewhat like a hack. I am wondering if the symfony maintainers plan to take on your suggestion, perhaps by offering a new config option to opt out of the error handler that defaults to true in the test environment.

Will this issue be resolved by symfony or is it expected to add @derrabus solution to every project?

This will be resolved by whomever resolves it. Could be you?

Thank you both for your patience with my silly question. I should have know, and think I did, but just forgot how the versioned branches work. Thanks again 😃

I stumbled over one test in @sulu which uses $client->disableReboot(); and the $client instanced is passed from one test to another in that case the shutdown is not called and so not cleaned up.

What I also tried is adding following to the KernelTestCase:

    /**
     * @internal
     */
    #[\PHPUnit\Framework\Attributes\After]
    public function __internalDisableErrorHandler(): void
    {
        \restore_exception_handler();
    }

But still has some issues with dataproviders in it.

Best result I got yet by adding it to the ensureKernelShutdown method:

    /**
     * Shuts the kernel down if it was used in the test - called by the tearDown method by default.
     */
    protected static function ensureKernelShutdown()
    {
        if (null !== static::$kernel) {
            static::$kernel->boot();
            $container = static::$kernel->getContainer();
            static::$kernel->shutdown();

            if ($container instanceof ResetInterface) {
                $container->reset();
            }
        }

        if (static::$booted) {
            static::$booted = false;
            \restore_exception_handler();
        }
    }

and in the WebTestCase:

    /**
     * @internal
     */
    #[\PHPUnit\Framework\Attributes\After]
    public function __internalDisableErrorHandler(): void
    {
        \restore_exception_handler();
    }

Strange is doing the __internalDisableErrorHandler in KernelTestCase makes the DataProviders still fail. Not doing __internalDisableErrorHandler makes test fails depending on $this->client->disableReboot();

that uses KernelTestCase or WebTestCase

this is not a root cause. I’d correct that to:

This issue is a blocker for PHPUnit 11 in any Symfony project that uses Kernel in tests.

E.g. for me the test maintenance is much easier when I’m not relying on SF testcases so I can extend my own leaner testcase. I use Kernel via composition in tests.

So the root cause is that there’s no way to cleanup after Kernel, not some test case.

@mariusjp see #53812 (comment)

Fair point, I’ve tried a few different scenarios. All are working fine, so can only conclude it’s a library that we are using that is interfering. Already tried a few, still can’t reproduce. I’ll let you know when I find something!

Can confirm @garak 's statement, just updated my project to Symfony 7.1.6;

After removing the hotfix:

require \dirname(__DIR__) . '/vendor/autoload.php';

// @todo: Remove temporary fix for the error handler for Symfony 7 in combination with PHPUnit 11
/* @see https://github.com/symfony/symfony/issues/53812 */
\set_exception_handler([new ErrorHandler(), 'handleException']);

I’m getting the same error again: “Test code or tested code did not remove its own exception handlers”

I’m running PHPUnit 11.4.2.

I believe my statement here is not fair, it’s very specific to my project, I can’t seem to reproduce the problem, see: https://github.com/symfony/symfony/issues/53812#issuecomment-2440802635

Yes, the issue still exists. And the workaround is documented in this thread.

After discussing this with @nikophil I found out that (by looking at https://github.com/sebastianbergmann/phpunit/blob/489040f5a752a47b3236f5998ec18a46e025f3d1/src/Framework/TestCase.php#L1907), only the exception_handler is an issue here as error handlers are correctly handled. My configuration has ignoreSuppressionOfDeprecations="true" as the trigger_deprecation method from Symfony uses the @ prefix.

I quite like the idea at https://github.com/symfony/symfony/pull/54266 but the code looks too complicated, basically what I did is:

    public function registerBundles(): array
    {
        $bundles = [
            new ApiPlatformBundle(),
            new FrameworkBundle(),
            new class extends Bundle
            {
                public function shutdown(): void
                {
                    restore_exception_handler();
                }
            }
        ];

I have a few suggestions to mitigate this issue:

  • add a $_SERVER variable ERROR_HANDLER_CLASS just as we have the KERNEL_CLASS allowing anyone to replace that if needed
  • have a ErrorHandler::unregister or a shutdown function that just calls restore_exception_handler but this looks too easy, @nicolas-grekas probably has use cases where it won’t work 😄
  • find out why the exception handler isn’t unregistered properly (or is registered too soon ?), it looks like the error handler is correctly removed according to phpunit’s code after each test
  • suggest an option on phpunit <source backupActiveExceptionListeners="false"> as IMO this should be de-activable on phpunit

uh, that explains why we keep missing out on deprecation notices. We use a lot of static setups that use the kernel

Damn, me too. Have I been unknowingly squashing deprecation warning for years 😳

I don’t know if both problems are related. If Symfony’s error handler is registered too soon, it would mute PHPUnit’s error handler 🤷

That’s weird. It’s the error handler that deals with deprecations, not the exception handler.

I know, but as soon as you boot the kernel in a “before class” hook no deprecation shows up…

For instance, this code does not display any deprecation:

final class SomeTest extends KernelTestCase
{
    public static function setUpBeforeClass(): void
    {
        self::bootKernel(); // <== if this line is removed, the deprecation is displayed
    }

    public function testWithDeprecation(): void
    {
        trigger_deprecation('some/package', '1.0', 'test deprecation.');
        self::assertTrue(true);
    }
}

Recently created a new Project with the Lumen Framework and wrote Tests for it and got this Warnings:

* Test code or tested code did not remove its own error handlers * Test code or tested code did not remove its own exception handlers

  • PHP 8.3.3
  • PhpUnit 11.0.6
  • Lumen 10.0.3

To overcome this Problem I needed to put this in the TestCase Class:


    protected function restoreExceptionHandler(): void
    {
        while (true) {
            $previousHandler = set_exception_handler(static fn() => null);
            restore_exception_handler();
            if ($previousHandler === null) {
                break;
            }
            restore_exception_handler();
        }
    }

    protected function restoreErrorHandler(): void
    {
        while (true) {
            $previousHandler = set_error_handler(static fn() => null);
            restore_error_handler();
            $isPhpUnitErrorHandler = ($previousHandler instanceof \PHPUnit\Runner\ErrorHandler);
            if ($previousHandler === null || $isPhpUnitErrorHandler) {
                break;
            }
            restore_error_handler();
        }
    }

    protected function tearDown(): void
    {
        parent::tearDown();
        $this->restoreErrorHandler();
        $this->restoreExceptionHandler();
    }

I hope this helps others, too.

Will this issue be resolved by symfony or is it expected to add @derrabus solution to every project?