sentry-laravel: ErrorException: Warning: PDO::prepare(): MySQL server has gone away

There 's a few tickets in the Laravel repositories, but it looks like the reason is an upgrade of the Sentry library to 1.0

Basically we all see ErrorException Warning: PDO::prepare(): MySQL server has gone away reports in Sentry, while it supposed to be caught by Laravel, and handle it.

But since the 1.0 update of Sentry library, it somehow bubbles up 😃

https://github.com/laravel/horizon/issues/583 https://github.com/laravel/framework/issues/28920

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Reactions: 10
  • Comments: 33 (3 by maintainers)

Most upvoted comments

Thanks for the info, maybe something with the log channel integration causing issues still. I know we fixed it when not using log channels. We’re looking!

@shengslogar I don’t think there is anything Laravel or Sentry can or should do.

I have discussed this with a few people and we think this is more a “bug” in PDO.

I’ll tell you why we think that.

When this error occurs “MySQL server has gone away” and others like it PDO throws an exception and generates an error with the E_WARNING level. There is no way in PHP to prevent a warning short of prefixing a function call with @ which is generally accepted to be a bad idea™. In my opinion if PDO throws the Exception the warning should not be generated. Laravel captures the Exception thrown and handles it gracefully but Sentry captures the error.

I want to emphasize that although Sentry says ErrorException it actually is an error (there is a difference). That is because in the SDK we convert the PHP error to an exception so we can log it to Sentry (hence the ErrorException, see: https://github.com/getsentry/sentry-php/blob/master/src/ErrorHandler.php#L346).


As for the “fix” of not sending it to Sentry, you can also use a class like this:

<?php

namespace App\Exceptions;

use Sentry\Event;
use Sentry\Severity;

class Sentry
{
    public static function before(Event $event): ?Event
    {
        if ($event->getLevel()->isEqualTo(Severity::warning())
            && !empty($exception = array_first($event->getExceptions()))
            && Str::contains($exception['value'] ?? '', [
                'Error while sending STMT_PREPARE packet',
                'server has gone away'
            ])) {
            return null;
        }

        return $event;
    }
}

And you can use this in your sentry.php config file like:

<?php

return [
    'before_send' => [App\Exceptions\Sentry::class, 'before'],
];

@Mult1Hunter I had the same issue; my guess is the App\Exceptions\Handler is not run on the queue. Here’s some experimental code I put in my config/sentry.php file successfully (see docs on before_send):

<?php

use Illuminate\Support\Str;
use Sentry\Event;

return [
    'before_send' => function (Event $event): ?Event {
        $exceptions = $event->getExceptions();

        // don't mess with large stacks
        if (\count($exceptions) === 1) {
            $exceptionMessage = $exceptions[0]['value'] ?? null;
            if (Str::contains($exceptionMessage, [
                'Error while sending STMT_PREPARE packet',
                'server has gone away'
            ])) {
                // skips exception
                return null;
            }
        }

        return $event;
    }
];

Obligatory disclaimer to any future copy-pasters that this will suppress even legitimate database issues, so keep whatever other notifications you have available to you on, or just deal with getting false positives.

CC @stayallive since you’re the only Sentry member so far. Is this being actively looked into?

Without going too far down the rabbit hole, I have an idea of where/why this is happening, but not necessarily the best way to solve this.

Whenever a query fails because of a lost connection (determined by the DetectsLostConnections trait), Laravel catches the exception and retries it – as others have stated. However, it always logs the query in the form of a QueryExecuted event. See here and here.

Sentry appropriately picks up on this (see here, ref-ing 1.1.0 since this is the version I’m on) as it always has, but I think the reason this is only being reported now is the addition of this block in recent versions (specifically the flushEvents method):

https://github.com/getsentry/sentry-laravel/blob/1e5f644b62ee73424ad8316e37b9a2dcf7290de8/src/Sentry/Laravel/EventHandler.php#L113-L119

This change was part of a1b66de5a629fcfd089d789de75244910912b6d7, explicitly added to improve queue reporting.

So technically Sentry is now reporting errors it always should’ve, but because of the way Laravel is logging failed queries, I don’t see any good way to distinguish between an actually failed query and a failed-but-successfully-retried query without making a change to Laravel’s logging (e.g. moving the logQuery method into the try{} block or something similar).

String matching through Sentry is a good “fix” for now, but this will obviously cause actual failed queries to be ignored.

I don’t know enough about the internals of either codebases to recommend a good, accepted solution, but hopefully this will give a jumpstart to any efforts to resolve this.

sentry/sentry-laravel 1.6.2 was just released which fixes the regression causing duplicate exceptions, thanks all for the information and patience!

Ahhh gotcha, this makes sense. Then I think the culprit is found. Will be fixed soon-ish (in the next few days).

I had the same issue with latest version

are you able to tell us a bit more about how you setup the Sentry package, are you using log channels and/or the snippet added to the App/Exceptions/Handler.php?

I use log channels alone

composer.lock

"name": "laravel/framework",
"version": "v6.10.1",
...
"name": "sentry/sentry-laravel",
"version": "1.5.0",

config/logging.php

'channels' => [
    'stack' => [
        'driver' => 'stack',
        'channels' => ['daily', 'sentry'],
        'ignore_exceptions' => false,
    ],

    'daily' => [
        'driver' => 'daily',
        'path' => storage_path('logs/laravel.log'),
        'level' => 'debug',
        'days' => 14,
    ],

    'sentry' => [
        'driver' => 'sentry',
    ],
],

This issue was closed however I never mentioned we think it’s fixed in 1.4.0, please do open it back up or open a new issue if the problem is not solved.

@stayallive Okay, I think I understand now.

So regarding the error_types option:

  • captureException method doesn’t apply the error_types filter – only the Sentry native error handler does
  • Laravel is using a similar error handler and is piping those all through the service provider which (assuming setup per Sentry docs) should be calling captureException (which would not include the PDO warning it catches)

…would I then be correct to assume most of the errors Sentry is natively receiving are redundant? And as a result (and related to your original point), I would be safe with even the most aggressive error_types filter as long as I was only concerned about Laravel-reported issues?

My last firing brain cell appreciates all your help.

It is possible that the PHP bug is this one: https://bugs.php.net/bug.php?id=63812

@Mult1Hunter The premise is that the same errors get thrown when there is a lost connection AND an actual database issue. Laravel is wired to retry a request once if it thinks it could be a lost connection, and then fail if it doesn’t work that time. Since we’re writing a blanket statement to suppress specific errors, we have no way of determining if it’s because there was a lost connection (and successfully reconnected) or if there’s actually a legitimate issue with your database. This is the same issue Sentry has internally which is creating this whole problem to begin with.

This is why if you don’t have separate database monitoring alerts available to you and you’re concerned about your database actually failing, I would recommend against implementing this solution and just deal with the false alerts.

@stayallive Thanks for your reply! So it sounds like at this point we should close out this issue and move towards making a PR to Laravel itself to not log “false” errors in this context.

@LasseRafn Nice! Right on. Totally forgot about the serializing closures thing.

While the above before_send solution might work, it will prevent config caching.

I suggest creating a service provider, and putting it in the boot method as follows:

\Sentry::getClient()->getOptions()->setBeforeSendCallback( function ( Event $event ): ?Event {
        $exceptions = $event->getExceptions();

        // don't mess with large stacks
        if (\count($exceptions) === 1) {
            $exceptionMessage = $exceptions[0]['value'] ?? null;
            if (Str::contains($exceptionMessage, [
                'Error while sending STMT_PREPARE packet',
                'server has gone away'
            ])) {
                // skips exception
                return null;
            }
        }

        return $event;
});

As far as we can tell short of error supressing database calls (putting the @ character in front of these calls: https://d.bouma.dev/3XfflIwWpJn0) the warning cannot be prevented even if the Exception that is also generated is caught by a framework like Laravel (which handles this warning gracefully and retries).

As far as the context below people seem to agree (although older answer) the only way to prevent this warning is to not capture warnings (which Sentry does do).

Context: https://stackoverflow.com/questions/34098828/php-pdo-exception-warning-on-mysql-has-gone-away?lq=1

I am not sure this is anything we can do apart from hardcode the warning supressing it from the SDK but that seems like a horrible band-aid fix that possibly will supress those warnings where it does matter.

Open to ideas!

@shengslogar Uff, nice implementation! Can you elaborate why this would silence legitimate database issues? To me it looks like this would only silence exceptions that contain 'Error while sending STMT_PREPARE packet', 'server has gone away'. Thanks for sharing, will give this a try!

@shengslogar, we convert it to an ErrorException (https://github.com/getsentry/sentry-php/blob/master/src/ErrorHandler.php#L346) because Sentry has no notion of a “error” and it has no real representation, so we use the (Silenced)ErrorException to encapsulate PHP errors. So changing the error_types will 100% work in this specific case 😃

But yes, this will silence all errors of the type “warning”.