yt-dlp: [youtube] Upload date being wrong by one day

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

  • I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

Please make sure the question is worded well enough to be understood

As I understand, I uploaded the video on 25-10-2021, but youtube processed it on 26-10-2021 therefore this date shows up instead of 25-10-2021? Reference: https://github.com/yt-dlp/yt-dlp/issues/7802

Command: yt-dlp --print upload_date “https://www.youtube.com/watch?v=OqjTtnmGv8s”

As I’ve seen there is no way to print the date after being processed by youtube that shows up near the description? Or do I understand it wrongly?

Provide verbose output that clearly demonstrates the problem

  • Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
  • If using API, add 'verbose': True to YoutubeDL params instead
  • Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['--print', 'upload_date', 'https://www.youtube.com/watch?v=OqjTtnmGv8s', '-vU']
[debug] Encodings: locale cp1250, fs utf-8, pref cp1250, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2024.04.09 from yt-dlp/yt-dlp [ff0779267] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.22631-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 7.0-essentials_build-www.gyan.dev (setts), ffprobe 7.0-essentials_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.02.02, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.31.0, sqlite3-3.35.5, urllib3-2.2.1, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets, curl_cffi
[debug] Loaded 1810 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: stable@2024.04.09 from yt-dlp/yt-dlp
yt-dlp is up to date (stable@2024.04.09 from yt-dlp/yt-dlp)
[youtube] Extracting URL: https://www.youtube.com/watch?v=OqjTtnmGv8s
[youtube] OqjTtnmGv8s: Downloading webpage
[youtube] OqjTtnmGv8s: Downloading ios player API JSON
[youtube] OqjTtnmGv8s: Downloading android player API JSON
WARNING: [youtube] Skipping player responses from android clients (got player responses for video "aQvGIIdgFDM" instead of "OqjTtnmGv8s")
[debug] Loading youtube-nsig.7ee5b648 from cache
[debug] [youtube] Decrypted nsig qBP4tfEIQJ6d9G23LXO => j7u7NJ33_M_BvQ
[debug] Loading youtube-nsig.7ee5b648 from cache
[debug] [youtube] Decrypted nsig iSggGPn2WbkdYE2hmMa => UclxAJpmlpI-iA
[youtube] OqjTtnmGv8s: Downloading m3u8 information
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] OqjTtnmGv8s: Downloading 1 format(s): 137+251
20211025

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 15 (10 by maintainers)

Commits related to this issue

Most upvoted comments

I’m confident the actual timestamp was not available the last time I checked. If we can now extract full timestamp instead of upload_date, that’s awesome! With timestamp, we don’t need non-UTC date since user can just add their timezone %(timestamp+xx>%Y%m%d)s as desired

Upstream gets the same result:

$ youtube-dl 'https://www.youtube.com/watch?v=OqjTtnmGv8s' --get-filename -o '%(upload_date)s'
20211025
$

That value is the 'uploadDate' in the page’s ytInitialPlayerResponse.

The page displayed by Chromium (Qt WebEngine) and by Firefox ESR (shows 2 years ago before expanding the description field) has:

73 views  25 Oct 2021
My discord recruiter, playing akali on bots in League of Legends game.
Mój rekruter discordowy, grający akali na boty w grze League of Legends.

From OP’s account, in CEST the page shows 26 instead of 25 (2021’s CEST ended on 31 October, the last Sunday of the month).

Why do we get 20211025? The yt-dl extractor is looking at the webpage:

(Pdb) pp re.findall(r'[^,>]{,45}2021-?10-?2\d[^\s>,]*', webpage)
['<meta itemprop="datePublished" content="2021-10-25T15:29:23-07:00"',
 '<meta itemprop="uploadDate" content="2021-10-25T15:29:23-07:00"',
 '"publishDate":"2021-10-25T15:29:23-07:00"',
 '"uploadDate":"2021-10-25T15:29:23-07:00"}}']
(Pdb) 

There is plenty of full resolution timestamp data there. The yt-dlp extractor looks at all the player responses that were both requested and available, but similar data also seemed to be present in the cases that I checked.

The root of the problem seems to be this:

  • the actual date-time is 2021-10-25T15:29:23-07:00(PDT) == 2021-10-26T00:29:23+02:00(CEST).
  • both extractors use the unified_strdate() function, essentially identical here, that has this special feature:
...
    # Remove AM/PM + timezone
    date_str = re.sub(r'(?i)\s*(?:AM|PM)(?:\s+[A-Z]+)?', '', date_str)
    _, date_str = extract_timezone(date_str)
...

Is it just me, or is this is complete nonsense?

The explanation might be, from the first commit of this function:

# %z (UTC offset) is only supported in python>=3.2

So this was a first pass that hasn’t been fixed since 2014. The extract_timezone() function was added that returns a datetime.timedelta for the TZ, but it was still discarded, while the TZ was used in parse_iso8601() and in unified_timestamp() (where it seems to be stripped twice?).

Solutions:

  • conservative but potentially confusing: leave the upload_date as-is (ie, the YYYYMMDD as measured at YT HQ) while adding a timestamp that is accurately based on UTC
  • more disruptively: scrap the explicit upload_date extraction in favour of a timestamp (that will result in an altered upload_date in cases like this), unless the page only offers YYYYMMDD.

Presumably, before JS got fancy date/time processing, YT used to send the YYYYMMDD resolution shown on the page, rather sending the ISO 8601 format and degrading it.