symfony: [Framework bundle] Cleanup error and exception handlers on Kernel shutdown
Symfony version(s) affected
7.0.0
Description
PHPUnit 11 checks for any leftovers in error handlers https://github.com/sebastianbergmann/phpunit/pull/5619
How to reproduce
Run FrameworkBundle->boot() in phpunit test.
It results in Test code or tested code did not remove its own exception handlers thrown by phpunit.
Possible Solution
Introduce FrameworkBundle::shutdown() that would cleanup error handler set in boot(). IMO it’s reasonable to have some cleanup in shutdown() of things initiated in boot().
Additional Context
No response
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 22
- Comments: 46 (32 by maintainers)
Commits related to this issue
- Update phpunit-bootstrap.php workaround for https://github.com/symfony/symfony/issues/53812#issuecomment-1962311843 — committed to TurboLabIt/webstackup by ZaneCEO 3 years ago
- chore(ci): use phpunit version 9 otherwise risky checks are warned https://github.com/symfony/symfony/issues/53812 — committed to whatwedo/MonitorBundle by xarem 3 years ago
- fix: Исправление ошибки Test code or tested code did not remove its own exception handlers Фикс взят из https://github.com/symfony/symfony/issues/53812 — committed to muratsalakhov/twitter-symfony by muratsalakhov 3 years ago
- Workaround Symfony issue #53812 (https://github.com/symfony/symfony/issues/53812) — committed to jgarciatorralba/presentation-portfolio-project-api by jgarciatorralba 2 years ago
- bug #58372 Tweak error/exception handler registration (nicolas-grekas) This PR was merged into the 5.4 branch. Discussion ---------- Tweak error/exception handler registration | Q | A ... — committed to symfony/symfony by nicolas-grekas 2 years ago
- Workaround for symfony/symfony#53812 — committed to smartassert/sources by webignition 2 years ago
- Workaround for symfony/symfony#53812 (#1468) — committed to smartassert/sources by webignition 2 years ago
- Apply workaround for symfony/symfony#53812 — committed to smartassert/job-coordinator by webignition 2 years ago
- Apply workaround for symfony/symfony#53812 (#832) — committed to smartassert/job-coordinator by webignition 2 years ago
- fix(tests): prevent phpunit to fails if the error handler has been replaced, see https://github.com/symfony/symfony/issues/53812 — committed to Kocal/hugo.alliau.me by Kocal 2 years ago
- Apply workaround for symfony/symfony#53812 — committed to smartassert/worker-manager by webignition 2 years ago
- Apply workaround for symfony/symfony#53812 (#567) — committed to smartassert/worker-manager by webignition 2 years ago
- api: pull setting exception handler up to boostrap.php As suggested in https://github.com/symfony/symfony/issues/53812#issuecomment-1962740145 This seems like a temporary fix, until symfony has figu... — committed to BacLuc/ecamp3 by BacLuc 2 years ago
- Update to PHPUnit 11 We were held back by https://github.com/symfony/symfony/issues/53812 The issue is that `FrameworkBundle` registers an exception handler (if `ErrorHandler` was not registered befo... — committed to wmde/fundraising-application by gbirke 2 years ago
A different workaround, seeing as the Symfony/Errorhandler is a singleton and will only register once, is to register the error handler before phpunit memorizes all the existing error handlers. This way the error handler Symfony register’s is not new.
Create (or modify)
/tests/bootstrap.php:and in
phpunit.xml@frankdekker is right, the error handler registered by FrameworkBundle would not replace the active error handler. PHPUnit already registers its own error handler and that one is still active at the end of the test. So we’re good here.
The problem which PHPUnit reports is about the exception handler. Yes, error handler and exception handler are two different things. And yes, it’s super confusing that Symfony has a method called
ErrorHandler::register()that registers an error handler and an exception handler. 😓PHPUnit on the other hand does not register an exception handler and even if it did, that handler would be wrapped by
ErrorHandler::register()which PHPUnit would still rightfully complain about.But.
ErrorHandlerwon’t replace the exception handler ifErrorHandleralready is the current exception handler. This means, we can simplify @frankdekker’s bootstrap script like this:That being said, this feels like a hack. Maybe we should add a way to opt out of the exception handler. In situations where the kernel is executed within a huge try/catch block (which a PHPUnit test basically is), that exception handler does not serve any purpose anyway.
I had the same problem with Symfony 6.4 and PHPUnit 11. I temporarily solved it by calling this method in the
tearDownof my base test case (which extends fromKernelTestCaseorWebTestCase):The
tearDownlooks like this:It is based on what Laravel did on their testsuite: https://github.com/laravel/framework/pull/49622/files
Maybe the solution is to call a similar method in the
tearDownofKernelTestCaseandWebTestCase?Same issue “Test code or tested code did not remove its own exception handlers” here with
Please, no more me-toos here. We know the issue exists and a workaround has been posted. Let’s focus on solutions.
What if we stopped registering error handlers, exception handlers or shutdown functions of any kind in FrameworkBundle?
We do need this stuff way before the framework is booted, so whatever FrameworkBundle registers here, it does it way too late. This applies especially to exception handlers: They are meant to handle uncaught exceptions. The kernel however runs in a big try/catch, so at the time FrameworkBundle is booted, the need for an exception handler is more or less over.
In modern Symfony applications, the error handling should be set up by the runtime already. All that FrameworkBundle does is connecting the logger properly. The logic that registers an error handler or exception handler is usually dormant and causes trouble because PHPUnit tests run outside of the runtime.
interesting.
this problem is absent on symfony 6.4, but exists in latest 7.2
This issue is a blocker for PHPUnit 11 in any Symfony project that uses
KernelTestCaseorWebTestCase. Given that PHPUnit comes with its own error handler, maybe the solution would be to opt out of Symfony’s own error handler when booting the kernel for functional tests?I’m using Laravel 11 and PHP Unit 11 and nothing fixes this issue. I can only run my tests on PHP Unit 10. Is anyone still experiencing this?
thanks very much @derrabus, you’re very simple addition to my pre-existing
tests/bootstrap.phpresolved the issue:I agree with you that, while simple, it does seem somewhat like a hack. I am wondering if the symfony maintainers plan to take on your suggestion, perhaps by offering a new config option to opt out of the error handler that defaults to
truein the test environment.This will be resolved by whomever resolves it. Could be you?
Thank you both for your patience with my silly question. I should have know, and think I did, but just forgot how the versioned branches work. Thanks again 😃
I stumbled over one test in @sulu which uses
$client->disableReboot();and the$clientinstanced is passed from one test to another in that case theshutdownis not called and so not cleaned up.What I also tried is adding following to the KernelTestCase:
But still has some issues with dataproviders in it.
Best result I got yet by adding it to the
ensureKernelShutdownmethod:and in the WebTestCase:
Strange is doing the __internalDisableErrorHandler in KernelTestCase makes the DataProviders still fail. Not doing
__internalDisableErrorHandlermakes test fails depending on$this->client->disableReboot();this is not a root cause. I’d correct that to:
E.g. for me the test maintenance is much easier when I’m not relying on SF testcases so I can extend my own leaner testcase. I use Kernel via composition in tests.
So the root cause is that there’s no way to cleanup after Kernel, not some test case.
Fair point, I’ve tried a few different scenarios. All are working fine, so can only conclude it’s a library that we are using that is interfering. Already tried a few, still can’t reproduce. I’ll let you know when I find something!
Can confirm @garak 's statement, just updated my project to Symfony 7.1.6;After removing the hotfix:I’m getting the same error again:“Test code or tested code did not remove its own exception handlers”I’m running PHPUnit 11.4.2.I believe my statement here is not fair, it’s very specific to my project, I can’t seem to reproduce the problem, see: https://github.com/symfony/symfony/issues/53812#issuecomment-2440802635
Yes, the issue still exists. And the workaround is documented in this thread.
After discussing this with @nikophil I found out that (by looking at https://github.com/sebastianbergmann/phpunit/blob/489040f5a752a47b3236f5998ec18a46e025f3d1/src/Framework/TestCase.php#L1907), only the
exception_handleris an issue here as error handlers are correctly handled. My configuration hasignoreSuppressionOfDeprecations="true"as thetrigger_deprecationmethod from Symfony uses the@prefix.I quite like the idea at https://github.com/symfony/symfony/pull/54266 but the code looks too complicated, basically what I did is:
I have a few suggestions to mitigate this issue:
$_SERVERvariableERROR_HANDLER_CLASSjust as we have theKERNEL_CLASSallowing anyone to replace that if neededErrorHandler::unregisteror ashutdownfunction that just callsrestore_exception_handlerbut this looks too easy, @nicolas-grekas probably has use cases where it won’t work 😄<source backupActiveExceptionListeners="false">as IMO this should be de-activable on phpunitDamn, me too. Have I been unknowingly squashing deprecation warning for years 😳
I don’t know if both problems are related. If Symfony’s error handler is registered too soon, it would mute PHPUnit’s error handler 🤷
I know, but as soon as you boot the kernel in a “before class” hook no deprecation shows up…
For instance, this code does not display any deprecation:
Recently created a new Project with the Lumen Framework and wrote Tests for it and got this Warnings:
* Test code or tested code did not remove its own error handlers * Test code or tested code did not remove its own exception handlers
To overcome this Problem I needed to put this in the TestCase Class:
I hope this helps others, too.
Will this issue be resolved by symfony or is it expected to add @derrabus solution to every project?