symfony: [ErrorHandler] Breaks json headers when updating to 4.4.1

Symfony version(s) affected: 4.4.1

Description

Hello I have a problem with the update to symfony 4.4.1 and the response headers. I am upgrading from 4.3.9 to 4.4.1

From my CI:

SendTemplateEmailControllerTest::whenVersionDoesNotExistThenItMustReturnNotFound
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'application/json'
+'application/x-www-form-urlencoded'

Line that fails:

        $this->assertSame('application/json', $response->headers->get('content-type'));

I got this error when i set up in twig.yaml according to UPGRADE.md:

exception_controller: null

I also tried in the framework.yaml:

    error_controller: 'error_controller'

But setting this inside framework.yaml seems to work again (make my tests pass):

error_controller: 'twig.controller.exception::showAction' this seems to work

Also my route contains this:

defaults={"domain"="%email_api_primary_domain%", "_format"="json"},

So what I expect is to get a json response in case of an error.

Extra context

TBH if this is a misconfiguration of mine I can not really how to fix this. I read this: https://symfony.com/blog/new-in-symfony-4-4-errorhandler-component?utm_source=feedburner&utm_medium=twitter&utm_campaign=Feed%3A+symfony%2Fblog+(Symfony+Blog)

And the changelog: https://github.com/symfony/symfony/blob/4.4/UPGRADE-4.4.md#twigbundle

And also tried slack channel for help… 😃