framework: Model toArray() with "date" or "datetime" casts serializes dates using timezone, whereas date casts with custom format don't

Laravel Version: 8.42.1
PHP Version: 7.4.5
Database Driver & Version: MySQL 8.0.18

Description:

When setting a timezone (for instance: APP_TIMEZONE="Europe/Brussels" in .env), all $model->toArray() calls will transform date and datetime casted attributes using the app’s timezone, unless the cast uses a custom datetime format such as datetime:Y-m-d H:i:s. This inconsistent behavior comes from the HasAttributes::serializeDate() method, which is called for the default date and datetime casts, but not for custom datetime casts (see lines 243 and 248).

IMO dates should never be automatically transformed using the .env’s APP_TIMEZONE for the following reasons :

Dates might already be stored correctly using the app’s timezone, in which case they don’t need to be transformed anymore. Assuming all dates are stored in UTC seems a little bit dangerous.
The default usage of date and datetime casts is not automatically timezone-transformed, meaning there will be inconsistencies between regular date usage in views such as $date->format('d.m.Y') and dates that are JSON serialized.

Steps To Reproduce:

Set your app’s .env’s timezone in something else than an UTC value, for instance :

APP_TIMEZONE="Europe/Brussels"

Create a migration containing an extra date column :

Schema::create('articles', function (Blueprint $table) {
    $table->id();
    $table->timestamp('published_at')->nullable();
    $table->timestamps();
});

Create a model :

namespace App\Models;

use Illuminate\Database\Eloquent\Model;

class Article extends Model
{
    protected $casts = [
        'published_at' => 'datetime',
    ];
}

Seed one or more database entries :

namespace Database\Seeders;

use App\Models\Article;
use Illuminate\Database\Seeder;

class ArticlesTableSeeder extends Seeder
{
    public function run()
    {
        Article::create([
            'published_at' => now()->startOfDay(),
        ]);
    }
}

Try the following route :

Route::get('/', function() {
    $article = \App\Models\Article::find(1);

    return [
        'formatted' => $article->published_at->format('Y-m-d H:i:s'),
        'model' => $article
    ];
});

It should return the following JSON :

Now, in your App\Models\Article model, change the datetime casting as follows :

class Article extends Model
{
    protected $casts = [
        'published_at' => 'datetime:Y-m-d H:i:s',
    ];
}

The JSON now contains the correct datetime (notice the 2 hours difference in model.published_at with the previous screenshot) :

About this issue

Original URL
State: closed
Created 3 years ago
Reactions: 1
Comments: 17 (13 by maintainers)

Most upvoted comments

@Nyratas so I talked to Kyle about all this on Discord. What the best solution for now is going to be is to document the behavior you’re experiencing. I’ll send in a PR for that.

To unblock you: you can override serializeDate in your models for now with the below so all of your serialized dates are in the same timezone as your app:

/**
 * Prepare a date for array / JSON serialization.
 *
 * @param  \DateTimeInterface  $date
 * @return string
 */
protected function serializeDate(DateTimeInterface $date)
{
    return Carbon::instance($date)->toISOString(true);
}

driesvints on Jun 1, 2021

This issue is in relation to upgrading to Laravel 9, one of the cases I faced was with that DB migration batch, but there are loads of other batches in the codebase that also use toArray with different use cases and its to cumbersome to go and check every case.

What i just did was extend a new BaseModel on all of my models and include this in my BaseModel to get back the old behavior of toArray.

protected function serializeDate(DateTimeInterface $date)
{
    return $date->format('Y-m-d H:i:s');
}

Anyone facing this issue, I located the upgrade guide where this is mentioned: https://laravel.com/docs/7.x/upgrade#date-serialization

bernardwiesner on Apr 6, 2022

Hi @driesvints, I just realized I didn’t come back to you in time when I posted this issue. Sorry about that and thank you very much for all the great work you and @kylekatarnls provided over the years (not only in this issue), it is very much appreciated! Keep up the good work guys 😃

toonvandenbos on Jan 25, 2022

@kylekatarnls One such use case would be migration to different DBs:

SomeBigTable::insert($records->toArray());

This would cause issues due to converting the dates to UTC with an error such as:

ERROR 1292: 1292: Incorrect datetime value: '2022-02-05T07:28:00.000000Z' for column 'created_at' at row 1
SQL Statement:
UPDATE `users` SET `created_at` = '2022-02-05T07:28:00.000000Z' WHERE (`id` = '1')

bernardwiesner on Apr 5, 2022

I know, it’s not the point, but first I wish to advice not having APP_TIMEZONE=“Europe/Brussels” or such kind of global application timezone. It’s even worse when the default timezone has DST, because when you will store 31 oct 2021 2h30 in your DB it can refer to 2 different moments (2h30 GMT+1 or 2h30 GMT+2, both exists in Brussel) and you will have absolutely now way to know from which offset it came when you retrieve it from the DB. For a query SELECT * FROM messages ORDER BY date DESC all messages sent before 2AM GMT+2 and 3AM GMT+2 will be mixed with those send in the next hour. + Plenty of other DST bugs coming directly from PHP: https://bugs.php.net/search.php?cmd=display&search_for=timezone&x=0&y=0

And it’s even even worse the this timezone is driven by an env var which means it can change if you change the env var and so you will get mixed timezones dates in your DB.

Last it does make any sense to store them with Brussels if you’re sending them as UTC anyway.

I detailed the position in this article: https://kylekatarnls.medium.com/always-use-utc-dates-and-times-8a8200ca3164

So be kind with your future you, stick your application timezone and DB timezone to UTC. You actually don’t need Europe/Brussel, UTC is just fine to represent universally a given instant making your database agnostic from where the server, company or any other stuff lives. It does not prevent to display dates in any timezone properly to the users.

For the tip having an already existing application is not a blocker, you can make it offline for maintenance, change date values then apply the new UTC settings so even previously stored dates will be correct.

Now, even if I don’t think Laravel should encourage to switch timezone at model level, this is still the business responsibility to be strict or not with this.

Carbon has a $keepOffset in toISOString (toISOString($keepOffset = false)) which is actually what needed here to get the Brussels offset used so $serializeDatesAsUTC like proposed by @driesvints could be easily linked in: $date->toISOString(!$this->serializeDatesAsUTC) but we still have an inconsistency or BC change as the offset is kept with the default JSON serialization but no longer as soon as a custom format is used.

So, a syntax like datetime:America/Toronto@Y-m-d H:i:s would be a bit better because:

it allows to use something else than config('app.timezone') or UTC
it allows to customize each property differently
default behavior can be more easily documented:
- if format is specified but not timezone, config('app.timezone') is used
- if timezone is specified but not the format, Y-m-d\TH:i:s.uP is used
- if none are specified, Y-m-d\TH:i:s.uP is used as format, UTC is used as timezone It does not get rid of the inconsistency but allow to control it in a single point. And it will be more easy on a next major version to enforce one or the other.

But there is a need to precise the syntax, datetime:Europe/Brussels:Y-m-d H:i:s is not possible because we can’t know if : is a separator between format/timezone or simply between units (like in H:i format side, or in GMT+02:00 timezone side), and we can’t detect what is a timezone or a format in a smart way, for instance GMT can be both a valid timezone or a valid date format. That’s why we need a dedicated separator such as @ in my example.

And we can replace empty string as timezone with config('app.timezone') so datetime:@Y-m-d H:i:s is Brussels with the given custom format in our example.

The same way, we could have the shortcut datetime:America/Toronto@ to set the timezone and keep the default format, but it would actualy be a small BC: currently 'datetime:' use "" as a date format. I can’t see any reason to do that so I don’t think, it would impact any real user but still worth to be known so if anyone has a better idea for a non-ambiguous backward-compatible syntax that would not prevent from using default values, it would be even better.

kylekatarnls on May 30, 2021

There’s no 2 hour difference, it’s just displayed for a different timezone. 22:00 in UTC and 00:00 in Europe/Brussels (CEST/UTC+2) are the same thing.

I’d recommend you always include the timezone (as is done in the default RFC 3339 output). That way, any consuming code doesn’t have to make any assumptions.

AegirLeet on May 24, 2021