yt-dlp: Tiktok: "Unable to extract json_string" from page within Europe

Checklist

Region

Global

Description

Issue

I keep getting an “Unable to extract json_string” error whilst downloading videos from TikTok, this only seems to be happening from countries that follow some type of GDPR (I’ve tried the UK, France, Spain and Germany). The video will download normally when a VPN is used to connect to a country that doesn’t follow GDPR (I’ve tried the US and Norway).

The only visible differene between TikTok in UK and the US is an “Accept Cookies” banner pictured below.

Images

These were taken on Firefox 95.0.2 on MacOS in a private window. UK Screenshot 2021-12-27 at 12 37 00 US Screenshot 2021-12-27 at 12 37 15

Verbose log

Logs without VPN turned on (UK)

joshua@Joshuas-MacBook-Pro accore % ./bin/ytdlp/bin/ytdlp-exec https://vm.tiktok.com/ZM8TtY45e/ --no-warnings --no-call-home --no-check-certificate --prefer-free-formats -Uv --referer https://www.tiktok.com
[debug] Command-line config: ['https://vm.tiktok.com/ZM8TtY45e/', '--no-warnings', '--no-call-home', '--no-check-certificate', '--prefer-free-formats', '-Uv', '--referer', 'https://www.tiktok.com']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, err utf-8, pref UTF-8
[debug] yt-dlp version 2021.12.27 [6223f67a8] (zip)
[debug] Python version 3.8.2 (CPython 64bit) - macOS-11.6-arm64-arm-64bit
[debug] exe versions: none
[debug] Optional libraries: sqlite
[debug] Proxy map: {}
Latest version: 2021.12.27, Current version: 2021.12.27
yt-dlp is up to date (2021.12.27)
[debug] [generic] Extracting URL: https://vm.tiktok.com/ZM8TtY45e/
[generic] ZM8TtY45e: Requesting header
[redirect] Following redirect to https://www.tiktok.com/@izukuliker/video/7045381460345376046?_d=secCgwIARCbDRjEFSACKAESPgo8QN%2FWIzZS1EDmFSUSVyEkS3LSLLXbF72tWomJqXTuQ7iokEPsCKzG4XDGxxxpGvkMGAVs%2FKQeUdde34dbGgA%3D&checksum=9e18bd299d718107518b1e817c36e343c20c49ad086a3ba2234638677a10d98c&language=en&preview_pb=0&sec_user_id=MS4wLjABAAAASJJokL4UlDlT9RfL8G7l5xJc7jAm0HXOkh2TwkR7eCtUDPB5PQ-Zs10gOZDzCj24&share_app_id=1233&share_item_id=7045381460345376046&share_link_id=0E981932-C908-4C6B-A532-2F86E97B4A42&source=h5_m&timestamp=1640434108&tt_from=copy&u_code=da6klg4d0bdm4e&user_id=6778203430994707461&utm_campaign=client_share&utm_medium=ios&utm_source=copy&_r=1
[debug] [TikTok] Extracting URL: https://www.tiktok.com/@izukuliker/video/7045381460345376046?_d=secCgwIARCbDRjEFSACKAESPgo8QN%2FWIzZS1EDmFSUSVyEkS3LSLLXbF72tWomJqXTuQ7iokEPsCKzG4XDGxxxpGvkMGAVs%2FKQeUdde34dbGgA%3D&checksum=9e18bd299d718107518b1e817c36e343c20c49ad086a3ba2234638677a10d98c&language=en&preview_pb=0&sec_user_id=MS4wLjABAAAASJJokL4UlDlT9RfL8G7l5xJc7jAm0HXOkh2TwkR7eCtUDPB5PQ-Zs10gOZDzCj24&share_app_id=1233&share_item_id=7045381460345376046&share_link_id=0E981932-C908-4C6B-A532-2F86E97B4A42&source=h5_m&timestamp=1640434108&tt_from=copy&u_code=da6klg4d0bdm4e&user_id=6778203430994707461&utm_campaign=client_share&utm_medium=ios&utm_source=copy&_r=1
[TikTok] 7045381460345376046: Downloading video details
[TikTok] 7045381460345376046: Downloading webpage
[TikTok] 7045381460345376046: Downloading video webpage
ERROR: [TikTok] 7045381460345376046: Unable to extract json_string; please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; type  yt-dlp -U  to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.; please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; type  yt-dlp -U  to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.
  File "./bin/ytdlp/bin/ytdlp-exec/yt_dlp/extractor/common.py", line 599, in extract
    ie_result = self._real_extract(url)
  File "./bin/ytdlp/bin/ytdlp-exec/yt_dlp/extractor/tiktok.py", line 368, in _real_extract
    json_string = self._search_regex(
  File "./bin/ytdlp/bin/ytdlp-exec/yt_dlp/extractor/common.py", line 1173, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)

Logs with VPN turned on (US)

joshua@Joshuas-MacBook-Pro accore % ./bin/ytdlp/bin/ytdlp-exec https://vm.tiktok.com/ZM8TtY45e/ --no-warnings --no-call-home --no-check-certificate --prefer-free-formats -Uv --referer https://www.tiktok.com
[debug] Command-line config: ['https://vm.tiktok.com/ZM8TtY45e/', '--no-warnings', '--no-call-home', '--no-check-certificate', '--prefer-free-formats', '-Uv', '--referer', 'https://www.tiktok.com']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, err utf-8, pref UTF-8
[debug] yt-dlp version 2021.12.27 [6223f67a8] (zip)
[debug] Python version 3.8.2 (CPython 64bit) - macOS-11.6-arm64-arm-64bit
[debug] exe versions: none
[debug] Optional libraries: sqlite
[debug] Proxy map: {}
Latest version: 2021.12.27, Current version: 2021.12.27
yt-dlp is up to date (2021.12.27)
[debug] [generic] Extracting URL: https://vm.tiktok.com/ZM8TtY45e/
[generic] ZM8TtY45e: Requesting header
[redirect] Following redirect to https://www.tiktok.com/@izukuliker/video/7045381460345376046?_d=secCgwIARCbDRjEFSACKAESPgo8QN%2FWIzZS1EDmFSUSVyEkS3LSLLXbF72tWomJqXTuQ7iokEPsCKzG4XDGxxxpGvkMGAVs%2FKQeUdde34dbGgA%3D&checksum=9e18bd299d718107518b1e817c36e343c20c49ad086a3ba2234638677a10d98c&language=en&preview_pb=0&sec_user_id=MS4wLjABAAAASJJokL4UlDlT9RfL8G7l5xJc7jAm0HXOkh2TwkR7eCtUDPB5PQ-Zs10gOZDzCj24&share_app_id=1233&share_item_id=7045381460345376046&share_link_id=0E981932-C908-4C6B-A532-2F86E97B4A42&source=h5_m&timestamp=1640434108&tt_from=copy&u_code=da6klg4d0bdm4e&user_id=6778203430994707461&utm_campaign=client_share&utm_medium=ios&utm_source=copy&_r=1
[debug] [TikTok] Extracting URL: https://www.tiktok.com/@izukuliker/video/7045381460345376046?_d=secCgwIARCbDRjEFSACKAESPgo8QN%2FWIzZS1EDmFSUSVyEkS3LSLLXbF72tWomJqXTuQ7iokEPsCKzG4XDGxxxpGvkMGAVs%2FKQeUdde34dbGgA%3D&checksum=9e18bd299d718107518b1e817c36e343c20c49ad086a3ba2234638677a10d98c&language=en&preview_pb=0&sec_user_id=MS4wLjABAAAASJJokL4UlDlT9RfL8G7l5xJc7jAm0HXOkh2TwkR7eCtUDPB5PQ-Zs10gOZDzCj24&share_app_id=1233&share_item_id=7045381460345376046&share_link_id=0E981932-C908-4C6B-A532-2F86E97B4A42&source=h5_m&timestamp=1640434108&tt_from=copy&u_code=da6klg4d0bdm4e&user_id=6778203430994707461&utm_campaign=client_share&utm_medium=ios&utm_source=copy&_r=1
[TikTok] 7045381460345376046: Downloading video details
[TikTok] 7045381460345376046: Downloading webpage
[TikTok] 7045381460345376046: Downloading video webpage
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: best/bestvideo+bestaudio
[info] 7045381460345376046: Downloading 1 format(s): 0
[debug] Invoking downloader on "https://v16-webapp.tiktok.com/ed9b36a30b4c7a8ee8efc584ee11c72a/61ca072f/video/tos/useast5/tos-useast5-ve-0068c002-tx/c204965a15c4499781bd6ef088e41581/?a=1988&br=2452&bt=1226&cd=0%7C0%7C1&ch=0&cr=0&cs=0&cv=1&dr=0&ds=3&er=&ft=Yu12_FGgkag3-I&l=2021122712341401018903602325043161&lr=tiktok_m&mime_type=video_mp4&net=0&pl=0&qs=0&rc=ajNoZDs6Zjg2OjMzZzczNEApaGg7Zmk3NGQ1NzVmNTNoM2dpNm5gcjRnZTBgLS1kMS9zc182LjMvNi4zNWM1Nl4uMjA6Yw%3D%3D&vl=&vr="
[download] #glamrockfreddy #fnaf [7045381460345376046].mp4 has already been downloaded
[download] 100% of 1.26MiB

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 5
  • Comments: 19 (4 by maintainers)

Commits related to this issue

Most upvoted comments

try running from master or wait for a new release @earthyrock the instructions to install from master are in the readme.

+3
Ashish0804 on Jan 7, 2022

WARNING: [TikTok] 7045687661859753222: Failed to parse JSON (caused by JSONDecodeError(‘Expecting value: line 1 column 1 (char 0)’)); please report this issue on https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; type yt-dlp -U to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.; Retrying with feed workaround

The issue is still there. We are just working around it with a fallback. Some devs are working on a proper solution. Hence why this issue is kept open

+3
pukkandan on Dec 29, 2021

@LastReaction wdym by latest build from the master repo? bc json_string doesn’t even exist on master branch anymore

I downloaded the zip file of the master branch from this repo and built the executable using pyinstaller… not sure what else to tell you but its still not working for me.

UPDATE: I’ve figured it out. It looks like the optional step of running the make_lazy_extractors.py script is what was causing the problem. I rebuilt the executable while removing that step and it now works fine. False alarm, sorry about that!

+1
GH0STDATA on Jan 12, 2022

I’m still getting the same issue with the latest build from the master repo. I’ve created a Windows executable using pyinstaller as instructed in the readme.

With the -F and verbose flags

yt-dlp -v -F "https://vm.tiktok.com/TTPdMkwX1c/"
[debug] Command-line config: ['-v', '-F', 'https://vm.tiktok.com/TTPdMkwX1c/']
[debug] User config file: C:\Users\<User>\AppData\Roaming\yt-dlp\config.txt
[debug] User config: ['-o', 'C:/Users/<User>/Downloads/yt-dlp/%(title)s.%(ext)s']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, err utf-8, pref cp1252
[debug] yt-dlp version 2021.12.27 [6223f67] (win_exe)
[debug] Python version 3.8.10 (CPython 64bit) - Windows-10-10.0.19044-SP0
[debug] exe versions: ffmpeg N-104277-gfb215798c7-20211008 (setts,fdk), ffprobe N-104277-gfb215798c7-20211008
[debug] Optional libraries: Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
[debug] [generic] Extracting URL: https://vm.tiktok.com/TTPdMkwX1c/
[generic] TTPdMkwX1c: Requesting header
[redirect] Following redirect to https://www.tiktok.com/@zai1k_/video/7049149340962573615?_d=secCgYIASAHKAESPgo8xIYsIXNJLIQmQofZm8t2VLZh38DQAcCnwVaRaI8YHpyxvMqAe0d%2FchPSTs7Q4uVJOahOvh8Xv1tX5wddGgA%3D&language=en&preview_pb=0&sec_user_id=MS4wLjABAAAAKfoJw1J7EMr4i7rlPm-paFqDIQ_7GhYiD1GNHz0Z94norBdGHmM1ZdJv7waiEjFg&share_app_id=1233&share_item_id=7049149340962573615&share_link_id=c35af0cc-d8ce-472b-98f9-fc8671cf0af8&source=h5_m&timestamp=1641579933&u_code=dj1hjm93gl8a15&user_id=6970543605769749510&utm_campaign=client_share&utm_medium=android&utm_source=copy&_r=1
[debug] [TikTok] Extracting URL: https://www.tiktok.com/@zai1k_/video/7049149340962573615?_d=secCgYIASAHKAESPgo8xIYsIXNJLIQmQofZm8t2VLZh38DQAcCnwVaRaI8YHpyxvMqAe0d%2FchPSTs7Q4uVJOahOvh8Xv1tX5wddGgA%3D&language=en&preview_pb=0&sec_user_id=MS4wLjABAAAAKfoJw1J7EMr4i7rlPm-paFqDIQ_7GhYiD1GNHz0Z94norBdGHmM1ZdJv7waiEjFg&share_app_id=1233&share_item_id=7049149340962573615&share_link_id=c35af0cc-d8ce-472b-98f9-fc8671cf0af8&source=h5_m&timestamp=1641579933&u_code=dj1hjm93gl8a15&user_id=6970543605769749510&utm_campaign=client_share&utm_medium=android&utm_source=copy&_r=1
[TikTok] 7049149340962573615: Downloading video details
WARNING: [TikTok] 7049149340962573615: Failed to parse JSON  (caused by JSONDecodeError('Expecting value: line 1 column 1 (char 0)')); please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; type  yt-dlp -U  to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.; Retrying with webpage
[TikTok] 7049149340962573615: Downloading webpage
[TikTok] 7049149340962573615: Downloading video webpage
ERROR: [TikTok] 7049149340962573615: Unable to extract json_string; please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; type  yt-dlp -U  to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.; please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; type  yt-dlp -U  to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.
  File "yt_dlp\extractor\common.py", line 599, in extract
    self._x_forwarded_for_ip = GeoUtils.random_ipv4(country)
  File "yt_dlp\extractor\tiktok.py", line 368, in _real_extract
    'duration': 18,
  File "yt_dlp\extractor\common.py", line 1173, in _search_regex
    if mobj:

With just the verbose flag

yt-dlp -v "https://vm.tiktok.com/TTPdMkwX1c/"
[debug] Command-line config: ['-v', 'https://vm.tiktok.com/TTPdMkwX1c/']
[debug] User config file: C:\Users\<User>\AppData\Roaming\yt-dlp\config.txt
[debug] User config: ['-o', 'C:/Users/<User>/Downloads/yt-dlp/%(title)s.%(ext)s']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, err utf-8, pref cp1252
[debug] yt-dlp version 2021.12.27 [6223f67] (win_exe)
[debug] Python version 3.8.10 (CPython 64bit) - Windows-10-10.0.19044-SP0
[debug] exe versions: ffmpeg N-104277-gfb215798c7-20211008 (fdk,setts), ffprobe N-104277-gfb215798c7-20211008
[debug] Optional libraries: Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
[debug] [generic] Extracting URL: https://vm.tiktok.com/TTPdMkwX1c/
[generic] TTPdMkwX1c: Requesting header
[redirect] Following redirect to https://www.tiktok.com/@zai1k_/video/7049149340962573615?_d=secCgYIASAHKAESPgo8xIYsIXNJLIQmQofZm8t2VLZh38DQAcCnwVaRaI8YHpyxvMqAe0d%2FchPSTs7Q4uVJOahOvh8Xv1tX5wddGgA%3D&language=en&preview_pb=0&sec_user_id=MS4wLjABAAAAKfoJw1J7EMr4i7rlPm-paFqDIQ_7GhYiD1GNHz0Z94norBdGHmM1ZdJv7waiEjFg&share_app_id=1233&share_item_id=7049149340962573615&share_link_id=c35af0cc-d8ce-472b-98f9-fc8671cf0af8&source=h5_m&timestamp=1641579933&u_code=dj1hjm93gl8a15&user_id=6970543605769749510&utm_campaign=client_share&utm_medium=android&utm_source=copy&_r=1
[debug] [TikTok] Extracting URL: https://www.tiktok.com/@zai1k_/video/7049149340962573615?_d=secCgYIASAHKAESPgo8xIYsIXNJLIQmQofZm8t2VLZh38DQAcCnwVaRaI8YHpyxvMqAe0d%2FchPSTs7Q4uVJOahOvh8Xv1tX5wddGgA%3D&language=en&preview_pb=0&sec_user_id=MS4wLjABAAAAKfoJw1J7EMr4i7rlPm-paFqDIQ_7GhYiD1GNHz0Z94norBdGHmM1ZdJv7waiEjFg&share_app_id=1233&share_item_id=7049149340962573615&share_link_id=c35af0cc-d8ce-472b-98f9-fc8671cf0af8&source=h5_m&timestamp=1641579933&u_code=dj1hjm93gl8a15&user_id=6970543605769749510&utm_campaign=client_share&utm_medium=android&utm_source=copy&_r=1
[TikTok] 7049149340962573615: Downloading video details
WARNING: [TikTok] 7049149340962573615: Failed to parse JSON  (caused by JSONDecodeError('Expecting value: line 1 column 1 (char 0)')); please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; type  yt-dlp -U  to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.; Retrying with webpage
[TikTok] 7049149340962573615: Downloading webpage
[TikTok] 7049149340962573615: Downloading video webpage
ERROR: [TikTok] 7049149340962573615: Unable to extract json_string; please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; type  yt-dlp -U  to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.; please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; type  yt-dlp -U  to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.
  File "yt_dlp\extractor\common.py", line 599, in extract
  File "yt_dlp\extractor\tiktok.py", line 368, in _real_extract
  File "yt_dlp\extractor\common.py", line 1173, in _search_regex
+1
GH0STDATA on Jan 11, 2022

wait for a new release

When that will be?

+1
f0rr357 on Jan 8, 2022

I opened “Issues w/ only TikTok #2250” but was immediately closed as a duplicate (cool, cool). Using Windows.exe, this is still an issue as of Jan 7. Any expectation of this issue being fixed in Windows.exe and/or a prospective release date? Thanks.

+1
earthyrock on Jan 7, 2022

can this be fixed or we have to use a vpn?

+1
ghost on Dec 28, 2021

The main difference appear to be whether you run the request via a known server host (VPN, VPS, dedicated hosting) or a home connection.

I ran -F on a tiktok video via Hetzner, Vultr, Edis and got the same error as OP. Running it from a private line in Sweden and Norway (both GDPR countries, via EU and EEC, respectively) gives a result but not the full list as before.

[debug] Command-line config: ['-v', '-F', 'https://www.tiktok.com/@girlinred/video/7003362805617298693']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, err utf-8, pref cp1252
[debug] yt-dlp version 2021.12.27 [6223f67] (win_exe)
[debug] Python version 3.8.10 (CPython 64bit) - Windows-10-10.0.19043-SP0
[debug] exe versions: ffmpeg 2021-10-24-git-9df3f147f5-full_build-www.gyan.dev (setts), ffprobe 2021-10-24-git-9df3f147f5-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
[debug] [TikTok] Extracting URL: https://www.tiktok.com/@girlinred/video/7003362805617298693
[TikTok] 7003362805617298693: Downloading video details
WARNING: [TikTok] 7003362805617298693: Failed to parse JSON  (caused by JSONDecodeError('Expecting value: line 1 column 1 (char 0)')); please report this issue on  https://github.com/yt-dlp/yt-dlp . Make sure you are using the latest version; type  yt-dlp -U  to update. Be sure to call yt-dlp with the --verbose flag and include its complete output.; Retrying with webpage
[TikTok] 7003362805617298693: Downloading webpage
[TikTok] 7003362805617298693: Downloading video webpage
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[info] Available formats for 7003362805617298693:
ID EXT RESOLUTION │ PROTO │ VCODEC  ACODEC
────────────────────────────────────────────
0  mp4 576x1024   │ https │ unknown unknown
+1
werid on Dec 27, 2021
Read more comments on GitHub
← yt-dlp: [TikTok] Failed to parse JSON/ No video formats found
yt-dlp: [tiktok:user] Failed to parse JSON →