symfony: [Mailer] Sending Messages Async with json serializer does not work

@Nyholm Personally I am not so sure this is Serializer’s issue. It looks to me like architectural mistake in Mailer/Mime component - there is SendEmailMessage message and even its name states it’s sending Email, not RawMessage which is used in signature. Not to mention RawMessage→Message inheritance which is weird (totally dropped parent class attributes, not called parent constructor). RawMessage seems to be unnatural abstraction layer to me. Code like this confirms my assumptions:

It’s from Symfony\Component\Mailer\Envelope, but there are other places with similar workarounds. RawMessage is mainly used for types in signatures while actual usage requires its child classes. Unfortunately it’s widely adopted and probably impossible to change at this point.

IMO Mailer/Notifier (and every other component that works with emails) should work with actual Email messages because it’s impossible to call Symfony\Component\Mailer\Mailer::send() with RawMessage (well, at least without exlipic Envelope which is not DX-friendly) - it will end up with LogicException regardless of how it’s handled internally by Mailer:

Without bus (using TransportInterface):

Event dispatcher:

Messenger bus:

Changing Mailer::send() and SendEmailMessage signatures and using Email would solve this issue and IMO would be generally better since it wouldn’t allow developers to get into that LogicException. For now, according to BC-promise, Mailer::send() could simply check if $message is an instance of Email and if not then it should check if $envelope is passed - if not, it could throw exception at this point, without calling transport/dispatcher/bus. It could also trigger deprecation warning and signature could be changed in the next major version. If Mailer::send() can’t create Email instance from RawMessage and Envelope, there probably should be DenormalizerInterface implementation for SendEmailMessage object so proper email message would be deserialized.

But it’s Symfony’s team decision of course 🙂 Maybe it was done this way for a reason I just don’t get.

The issue still exists. I had to use the NativePhpSerializer to make it work.

serializer: messenger.transport.native_php_serializer

I did an ugly trick with serializer, because SendEmailMessage structure is too complicated for json. The special serializer was added to a project

<?php

namespace Fxbo\Serializer;

use Symfony\Component\Mailer\Messenger\SendEmailMessage;
use Symfony\Component\Serializer\Normalizer\ContextAwareDenormalizerInterface;
use Symfony\Component\Serializer\Normalizer\ContextAwareNormalizerInterface;

class SendEmailMessageJsonSerializer implements ContextAwareNormalizerInterface, ContextAwareDenormalizerInterface
{
    public function supportsNormalization($data, $format = null, array $context = [])
    {
        return $data instanceof SendEmailMessage && $format === 'json';
    }

    public function normalize($object, $format = null, array $context = [])
    {
        return [__CLASS__ => addslashes(serialize($object))];
    }

    public function supportsDenormalization($data, $type, $format = null, array $context = [])
    {
        return $type == SendEmailMessage::class && $format === 'json' && isset($data[__CLASS__]);
    }

    public function denormalize($data, $type, $format = null, array $context = [])
    {
        return unserialize(stripslashes($data[__CLASS__]));
    }
}

And one more, I did not find how to inject context to serializer for specific class, so there is a file for exclude attributes:

<?php

namespace Fxbo\Serializer;

use Symfony\Component\Debug\Exception\FlattenException;
use Symfony\Component\Serializer\Normalizer\AbstractObjectNormalizer;
use Symfony\Component\Serializer\Normalizer\ContextAwareDenormalizerInterface;
use Symfony\Component\Serializer\Normalizer\ContextAwareNormalizerInterface;
use Symfony\Component\Serializer\Normalizer\ObjectNormalizer;
use Symfony\Component\Serializer\SerializerInterface;

class FlattenExceptionSerializer implements ContextAwareNormalizerInterface, ContextAwareDenormalizerInterface
{
    /**
     * @var ObjectNormalizer
     */
    private $normalizer;

    public function __construct(ObjectNormalizer $normalizer)
    {
        $this->normalizer = $normalizer;
    }

    public function supportsNormalization($data, $format = null, array $context = [])
    {
        return $data instanceof FlattenException;
    }

    public function normalize($object, $format = null, array $context = [])
    {
        return $this->normalizer->normalize(
            $object,
            $format,
            array_merge($context, [AbstractObjectNormalizer::IGNORED_ATTRIBUTES => ['previous', 'allPrevious']])
        );
    }

    public function supportsDenormalization($data, $type, $format = null, array $context = [])
    {
        return $type == FlattenException::class;
    }

    public function denormalize($data, $type, $format = null, array $context = [])
    {
        return $this->normalizer->denormalize(
            $data,
            $type,
            $format,
            array_merge($context, [AbstractObjectNormalizer::IGNORED_ATTRIBUTES => ['previous', 'allPrevious']])
        );
    }
}

Firstly I added a specific route and serializer, but there is an error when SendEmailMessage goes to failed queue it decoded as json and this task never be processed successfully.

framework:
    messenger:
        serializer:
            default_serializer: messenger.transport.symfony_serializer
            symfony_serializer:
                format: json
        # after retrying, messages will be sent to the "failed" transport
        failure_transport: failed
        transports:
            mailer:
                dsn: "%env(MESSENGER_TRANSPORT_DSN)%"
                serializer: 'messenger.transport.native_php_serializer'
                options:
                    queue_name: mailer
                    auto_setup: false
            failed:
                dsn: '%env(MESSENGER_TRANSPORT_DSN)%'
                options:
                    queue_name: failed
                    auto_setup: false
            #...
        routing:
            'Symfony\Component\Mailer\Messenger\SendEmailMessage':  mailer

@fabpot please, could you explain why is it designed this way ( https://github.com/symfony/symfony/issues/33394#issuecomment-791295930 ) ? 🙏 as you are the author of these classes/logic. understanding it will be a good start to wrap around this issue. notably

• the inheritance RawMessage -> Message -> Email
• most (all?) consumer methods are typed to RawMessage
• Email is a child class of RawMessage but totally overrides its constructor
• RawMessage is mainly used for types in signatures

seems to me that RawMessage is kinda substitute for an interface.

not actually solved by the related PR, as it was fixing only the case of serialize($rawMessage)