symfony: [Mailer] Sending Messages Async with json serializer does not work

Symfony version(s) affected: symfony/mailer: 4.3.4

Description
Sending Messages Async with json serializer does not work with SendEmailMessage.

Symfony\Component\Serializer\Exception\MissingConstructorArgumentsException : Cannot create an instance of Symfony\Component\Mime\RawMessage from serialized data because its constructor requires parameter "message" to be present.
/home/vagrant/www/fxbackoffice.local/vendor/symfony/serializer/Normalizer/AbstractNormalizer.php:505
 ...

How to reproduce

framework:
    messenger:
        serializer:
            default_serializer: messenger.transport.symfony_serializer
            symfony_serializer:
                format: json
        transports:
            async: "%env(MESSENGER_TRANSPORT_DSN)%"

        routing:
            'Symfony\Component\Mailer\Messenger\SendEmailMessage':  async

About this issue

  • Original URL
  • State: open
  • Created 7 years ago
  • Reactions: 23
  • Comments: 37 (21 by maintainers)

Commits related to this issue

Most upvoted comments

@Nyholm Personally I am not so sure this is Serializer’s issue. It looks to me like architectural mistake in Mailer/Mime component - there is SendEmailMessage message and even its name states it’s sending Email, not RawMessage which is used in signature. Not to mention RawMessage→Message inheritance which is weird (totally dropped parent class attributes, not called parent constructor). RawMessage seems to be unnatural abstraction layer to me. Code like this confirms my assumptions:

image

It’s from Symfony\Component\Mailer\Envelope, but there are other places with similar workarounds. RawMessage is mainly used for types in signatures while actual usage requires its child classes. Unfortunately it’s widely adopted and probably impossible to change at this point.

IMO Mailer/Notifier (and every other component that works with emails) should work with actual Email messages because it’s impossible to call Symfony\Component\Mailer\Mailer::send() with RawMessage (well, at least without exlipic Envelope which is not DX-friendly) - it will end up with LogicException regardless of how it’s handled internally by Mailer:

Without bus (using TransportInterface): image

Event dispatcher: image

Messenger bus: image

Changing Mailer::send() and SendEmailMessage signatures and using Email would solve this issue and IMO would be generally better since it wouldn’t allow developers to get into that LogicException. For now, according to BC-promise, Mailer::send() could simply check if $message is an instance of Email and if not then it should check if $envelope is passed - if not, it could throw exception at this point, without calling transport/dispatcher/bus. It could also trigger deprecation warning and signature could be changed in the next major version. If Mailer::send() can’t create Email instance from RawMessage and Envelope, there probably should be DenormalizerInterface implementation for SendEmailMessage object so proper email message would be deserialized.

But it’s Symfony’s team decision of course 🙂 Maybe it was done this way for a reason I just don’t get.

Any update on this one? It seems kind of counter intuitive, having to configure an extra transport (which includes a separate failure transport, too) with native PHP Serializer, while all other transports are working with JSON. On the other hand, when using SendEmailMessage, you are probably consuming the Mail with Symfony itself. Don’t get me wrong, just wondering if this issue is a won’t fix or needs support 😃

The issue still exists. I had to use the NativePhpSerializer to make it work.

serializer: messenger.transport.native_php_serializer

As rapid fix, I reused the Normalizer provided by @TheRatG (many thanks to them!) to work around this issue. I added a AutoconfigureTag to enable it faster. Tested on Sf 7.4.

namespace App\Serializer;

use Symfony\Component\DependencyInjection\Attribute\AutoconfigureTag;
use Symfony\Component\Mailer\Messenger\SendEmailMessage;
use Symfony\Component\Serializer\Normalizer\DenormalizerInterface;
use Symfony\Component\Serializer\Normalizer\NormalizerInterface;

#[AutoconfigureTag('serializer.normalizer', ['priority' => 500])]
class SendEmailMessageDenormalizer implements NormalizerInterface, DenormalizerInterface
{
    public function supportsNormalization($data, $format = null, array $context = []): bool
    {
        return $data instanceof SendEmailMessage && $format === 'json';
    }

    public function normalize($data, $format = null, array $context = []): float|int|bool|\ArrayObject|array|string|null
    { return [__CLASS__ => addslashes(serialize($data))];
    }

    public function supportsDenormalization($data, $type, $format = null, array $context = []): bool
    {
        return $type == SendEmailMessage::class && $format === 'json' && isset($data[__CLASS__]);
    }

    public function denormalize($data, $type, $format = null, array $context = []): mixed
    {
        return unserialize(stripslashes($data[__CLASS__]));
    }

    public function getSupportedTypes(?string $format): array
    {
        return [SendEmailMessage::class => true];
    }
}

What would be the best long-term solution here?

Should we consider revisiting the RawMessage class so it can be natively supported by the Symfony serializer? That could imply deprecating it in its current form and introducing a new class that would be fully supported wherever RawMessage is currently used.

Alternatively, could introducing a few interfaces help make things more flexible and extensible?

Or would it make more sense to simply include this Normalizer directly in the Mail component?

I’d love to hear your thoughts on the most sustainable path forward.

@fabpot please, could you explain why is it designed this way ( https://github.com/symfony/symfony/issues/33394#issuecomment-791295930 ) ? 🙏 as you are the author of these classes/logic. understanding it will be a good start to wrap around this issue. notably

  • the inheritance RawMessage -> Message -> Email
  • most (all?) consumer methods are typed to RawMessage
  • Email is a child class of RawMessage but totally overrides its constructor
  • RawMessage is mainly used for types in signatures

seems to me that RawMessage is kinda substitute for an interface.

I did an ugly trick with serializer, because SendEmailMessage structure is too complicated for json. The special serializer was added to a project

<?php

namespace Fxbo\Serializer;

use Symfony\Component\Mailer\Messenger\SendEmailMessage;
use Symfony\Component\Serializer\Normalizer\ContextAwareDenormalizerInterface;
use Symfony\Component\Serializer\Normalizer\ContextAwareNormalizerInterface;

class SendEmailMessageJsonSerializer implements ContextAwareNormalizerInterface, ContextAwareDenormalizerInterface
{
    public function supportsNormalization($data, $format = null, array $context = [])
    {
        return $data instanceof SendEmailMessage && $format === 'json';
    }

    public function normalize($object, $format = null, array $context = [])
    {
        return [__CLASS__ => addslashes(serialize($object))];
    }

    public function supportsDenormalization($data, $type, $format = null, array $context = [])
    {
        return $type == SendEmailMessage::class && $format === 'json' && isset($data[__CLASS__]);
    }

    public function denormalize($data, $type, $format = null, array $context = [])
    {
        return unserialize(stripslashes($data[__CLASS__]));
    }
}

And one more, I did not find how to inject context to serializer for specific class, so there is a file for exclude attributes:

<?php

namespace Fxbo\Serializer;

use Symfony\Component\Debug\Exception\FlattenException;
use Symfony\Component\Serializer\Normalizer\AbstractObjectNormalizer;
use Symfony\Component\Serializer\Normalizer\ContextAwareDenormalizerInterface;
use Symfony\Component\Serializer\Normalizer\ContextAwareNormalizerInterface;
use Symfony\Component\Serializer\Normalizer\ObjectNormalizer;
use Symfony\Component\Serializer\SerializerInterface;

class FlattenExceptionSerializer implements ContextAwareNormalizerInterface, ContextAwareDenormalizerInterface
{
    /**
     * @var ObjectNormalizer
     */
    private $normalizer;

    public function __construct(ObjectNormalizer $normalizer)
    {
        $this->normalizer = $normalizer;
    }

    public function supportsNormalization($data, $format = null, array $context = [])
    {
        return $data instanceof FlattenException;
    }

    public function normalize($object, $format = null, array $context = [])
    {
        return $this->normalizer->normalize(
            $object,
            $format,
            array_merge($context, [AbstractObjectNormalizer::IGNORED_ATTRIBUTES => ['previous', 'allPrevious']])
        );
    }

    public function supportsDenormalization($data, $type, $format = null, array $context = [])
    {
        return $type == FlattenException::class;
    }

    public function denormalize($data, $type, $format = null, array $context = [])
    {
        return $this->normalizer->denormalize(
            $data,
            $type,
            $format,
            array_merge($context, [AbstractObjectNormalizer::IGNORED_ATTRIBUTES => ['previous', 'allPrevious']])
        );
    }
}

Firstly I added a specific route and serializer, but there is an error when SendEmailMessage goes to failed queue it decoded as json and this task never be processed successfully.

framework:
    messenger:
        serializer:
            default_serializer: messenger.transport.symfony_serializer
            symfony_serializer:
                format: json
        # after retrying, messages will be sent to the "failed" transport
        failure_transport: failed
        transports:
            mailer:
                dsn: "%env(MESSENGER_TRANSPORT_DSN)%"
                serializer: 'messenger.transport.native_php_serializer'
                options:
                    queue_name: mailer
                    auto_setup: false
            failed:
                dsn: '%env(MESSENGER_TRANSPORT_DSN)%'
                options:
                    queue_name: failed
                    auto_setup: false
            #...
        routing:
            'Symfony\Component\Mailer\Messenger\SendEmailMessage':  mailer

not actually solved by the related PR, as it was fixing only the case of serialize($rawMessage)