yt-dlp: [FujiTVFODPlus7] Failed to parse JSON / FOD FujiTV

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

  • I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

Region

Japan

Provide a description that is worded well enough to be understood

[FujiTVFODPlus7] 2752810042: 2752810042: Failed to parse JSON (caused by JSONDecodeError(“Expecting value in ‘’: line 1 column 1 (char 0)”)); please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U File “yt_dlp\extractor\common.py”, line 734, in extract File “yt_dlp\extractor\fujitv.py”, line 50, in _real_extract File “yt_dlp\extractor\common.py”, line 1133, in download_content File “yt_dlp\extractor\common.py”, line 1100, in download_handle File “yt_dlp\extractor\common.py”, line 1088, in parse File “yt_dlp\extractor\common.py”, line 1075, in _parse_json File “yt_dlp\extractor\common.py”, line 1058, in __print_error

File “yt_dlp\utils_utils.py”, line 552, in decode File “json\decoder.py”, line 337, in decode File “json\decoder.py”, line 355, in raw_decode json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File “yt_dlp\extractor\common.py”, line 1072, in parse_json File "json_init.py", line 370, in loads File “yt_dlp\utils_utils.py”, line 560, in decode json.decoder.JSONDecodeError: Expecting value in ‘’: line 1 column 1 (char 0)

Provide verbose output that clearly demonstrates the problem

  • Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
  • If using API, add 'verbose': True to YoutubeDL params instead
  • Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

D:\yt-dlp_win>yt-dlp https://fod.fujitv.co.jp/title/2752/2752810042/ -vU
[debug] Command-line config: ['https://fod.fujitv.co.jp/title/2752/2752810042/', '-vU']
[debug] Encodings: locale cp932, fs utf-8, pref cp932, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2024.05.27 from yt-dlp/yt-dlp [12b248ce6] (win32_dir)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.22631-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 2022-04-07-git-607ecc27ed-full_build-www.gyan.dev (setts), ffprobe 2022-04-07-git-607ecc27ed-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.02.02, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.32.2, sqlite3-3.35.5, urllib3-2.2.1, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets, curl_cffi
[debug] Loaded 1820 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: stable@2024.05.27 from yt-dlp/yt-dlp
yt-dlp is up to date (stable@2024.05.27 from yt-dlp/yt-dlp)
[FujiTVFODPlus7] Extracting URL: https://fod.fujitv.co.jp/title/2752/2752810042/
[FujiTVFODPlus7] 2752810042: Downloading webpage
[FujiTVFODPlus7] 2752810042: Downloading JSON metadata
[FujiTVFODPlus7] 2752810042: Downloading JSON metadata
ERROR: [FujiTVFODPlus7] 2752810042: 2752810042: Failed to parse JSON (caused by JSONDecodeError("Expecting value in '': line 1 column 1 (char 0)")); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "yt_dlp\extractor\common.py", line 734, in extract
  File "yt_dlp\extractor\fujitv.py", line 50, in _real_extract
  File "yt_dlp\extractor\common.py", line 1133, in download_content
  File "yt_dlp\extractor\common.py", line 1100, in download_handle
  File "yt_dlp\extractor\common.py", line 1088, in parse
  File "yt_dlp\extractor\common.py", line 1075, in _parse_json
  File "yt_dlp\extractor\common.py", line 1058, in __print_error

  File "yt_dlp\utils\_utils.py", line 552, in decode
  File "json\decoder.py", line 337, in decode
  File "json\decoder.py", line 355, in raw_decode
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "yt_dlp\extractor\common.py", line 1072, in _parse_json
  File "json\__init__.py", line 370, in loads
  File "yt_dlp\utils\_utils.py", line 560, in decode
json.decoder.JSONDecodeError: Expecting value in '': line 1 column 1 (char 0)

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 24 (11 by maintainers)

Most upvoted comments

Managed to intercept sp_android, it is the same as browser only up to 720p. Also, tv_android doesn’t work, I think they are using a different name for android tv. Also, abrjson_v2 didn’t change, it still follow: https://i.fod.fujitv.co.jp/abrjson_v2/sp_android/(EP ID)?t={long code}

Maybe they have changed it again. When I tested the API when my code failed the first time, what I intercepted was the pc_html5 API, it appeared the https://i.fod.fujitv.co.jp/abrjson_v2/sp_android/(EP ID)/{some hash} format.

The extractor used to use the sp_android API, which I tested when I developed the internal tool for my fansub, returns 720P, then I changed that to use tv_android which returns 1080P, which is no longer working due to the URL scheme change, causing this issue.

*I just tested the sp_android API just now, and it doesn’t work for the latest episode, which is playable on the front end.

Now the URL directly returns from https://fod.fujitv.co.jp/apps/api/1/auth/contents/web?site_id=fodapp&ep_id=5d40810217&qa=auto&uuid=x5w61dsg-yjwb-4ito-2f35-o8n3ujrabu3d&starttime=0&is_pt=true&dt=&_=1727790729564

Full request is attached as follows

curl 'https://fod.fujitv.co.jp/apps/api/1/auth/contents/web?site_id=fodapp&ep_id=5d40810217&qa=auto&uuid=x5w61dsg-yjwb-4ito-2f35-o8n3ujrabu3d&starttime=0&is_pt=true&dt=&_=1727790729564' \
  -H 'accept: */*' \
  -H 'accept-language: zh-TW,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6,zh-CN;q=0.5' \
  -H 'cookie: fod_player_playbackRate=1.0; fod_player_subSound=0; fod_player_cc=0; fod_player_dub=0; fod_player_continuPlay=1; fod_player_fastForward=30; fod_player_quality=0; fod_player_rewind=30; undefined=bc53ffa1f06b616163f28bc28d508ed9; AMP_MKTG_fe4beb374f=JTdCJTIycmVmZXJyZXIlMjIlM0ElMjJodHRwcyUzQSUyRiUyRnd3dy5iaW5nLmNvbSUyRiUyMiUyQyUyMnJlZmVycmluZ19kb21haW4lMjIlM0ElMjJ3d3cuYmluZy5jb20lMjIlN0Q=; dv=; ssid=7wFVmaAX76F6vjEguf7Da7H7IQ+uyGdOXDa7kwu2lbE=; uuid=x5w61dsg-yjwb-4ito-2f35-o8n3ujrabu3d; ontouchend_flg=false; plus7_ans=202104; plus7_guid=565a8234-6741-4993-9c6d-3d092a13bd0d; plus7_attr=0_1965-05-01_1378088; plus7_ct=131032; CT=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJGT0QiLCJ1aWQiOiIiLCJkdl90eXBlIjoid2ViIiwiZHZfaWQiOiJNaWNyb3NvZnQgV2luZG93cyBOVCAxMC4wLjE3NzYzLjBfQ2hyb21lMTI5LjAifQ.o4DYHBF-JjMIyZpvozoxGVbKgg0qTQspsQnzS6FXNN4; UT=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJGT0QiLCJ1aWQiOiIiLCJkdl90eXBlIjoid2ViIiwiZHZfaWQiOiJNaWNyb3NvZnQgV2luZG93cyBOVCAxMC4wLjE3NzYzLjBfQ2hyb21lMTI5LjAifQ.rSnx7-Aw764yDDVMnmGKN_n2auJRdLS7awQZb-1GL1w; LC=0; ruid=ix9RhxuMKHF/kirShJ170A==; AMP_fe4beb374f=JTdCJTIyZGV2aWNlSWQlMjIlM0ElMjIxOTU1OGZkNy1kNmIxLTRiNTktYTVmOC1kMzU4NWJjNWYyMGQlMjIlMkMlMjJ1c2VySWQlMjIlM0ElMjIxNGM2ZWQ1My05NzBkLTQ0YWYtODFlZi1lN2I4NDNjZWEyODQlMjIlMkMlMjJzZXNzaW9uSWQlMjIlM0ExNzI3NzkwMzI1ODUyJTJDJTIyb3B0T3V0JTIyJTNBZmFsc2UlMkMlMjJsYXN0RXZlbnRUaW1lJTIyJTNBMTcyNzc5MDMyNTg1OCUyQyUyMmxhc3RFdmVudElkJTIyJTNBMCU3RA==; d6hkt=2ff5f2ab-fe80-4879-9b2d-a2ef53da04ff' \
  -H 'priority: u=1, i' \
  -H 'referer: https://fod.fujitv.co.jp/title/5d40/5d40810217' \
  -H 'sec-ch-ua: "Microsoft Edge";v="129", "Not=A?Brand";v="8", "Chromium";v="129"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "Windows"' \
  -H 'sec-fetch-dest: empty' \
  -H 'sec-fetch-mode: cors' \
  -H 'sec-fetch-site: same-origin' \
  -H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36 Edg/129.0.0.0' \
  -H 'x-authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJGT0QiLCJ1aWQiOiIiLCJkdl90eXBlIjoid2ViIiwiZHZfaWQiOiJNaWNyb3NvZnQgV2luZG93cyBOVCAxMC4wLjE3NzYzLjBfQ2hyb21lMTI5LjAifQ.rSnx7-Aw764yDDVMnmGKN_n2auJRdLS7awQZb-1GL1w' \
  -H 'x-requested-with: XMLHttpRequest'

Hope the latest information helps. Btw, any Japan VPN would circumvent this geo-lock.

sorry for Chinese comments tho, this is the internal tool that my fansub use to rip fujitv, which still works till this day, maybe it doesn’t download 1080P, but at least it works

#!/bin/bash
# check if jq、streamlink、ffmpeg、MP4Box and curl is installed, if not install automatically
if ! [ -x "$(command -v jq)" ]; then
  echo 'error jq is not installed' >&2
  sudo apt install jq
  exit 1
fi

if ! [ -x "$(command -v streamlink)" ]; then
  echo 'error streamlink is not installed' >&2
  sudo apt install streamlink
  exit 1
fi

if ! [ -x "$(command -v ffmpeg)" ]; then
  echo 'error ffmpeg is not installed' >&2
  sudo apt install ffmpeg
  exit 1
fi

if ! [ -x "$(command -v MP4Box)" ]; then
  echo 'error MP4Box is not installed' >&2
  sudo apt install gpac
  exit 1
fi

if ! [ -x "$(command -v curl)" ]; then
  echo 'error curl is not installed' >&2
  sudo apt install curl
  exit 1
fi



# get series_id、selected and selection_end from user input or use default value
#read -p "输入 series_id (预设: 5d40):" series_id
#read -p "输入 selected episode (预设: -1):" selected
#read -p "输入 selected end episode (预设: -1): " selected_end

# set default
series_id=${series_id:-'5d40'}
selected=${selected:-'-1'}
selected_end=${selected_end:-'-1'}

API_URL='https://i.fod.fujitv.co.jp'
FRONT_END='https://fod.fujitv.co.jp'



# eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJGT0QiLCJ1aWQiOiIiLCJkdl90eXBlIjoid2ViIiwiZHZfaWQiOiJNaWNyb3NvZnQgV2luZG93cyBOVCAxMC4wLjE0MzkzLjBfVW5rbm93bjAuMCJ9.j6DkHr_gtjUF8JHSy0rA8_e-rJOpfJKDFKcmqBGUk-Q

# get cookie and ignore html
token=$(curl -s -I $FRONT_END | awk -F'[:;]' '/CT/ {gsub(/^ */, "", $2); gsub(/ *$/, "", $2); if ($2) {sub(/^CT=/, "", $2); print $2}}')


json_response=$(curl -s -H "x-authorization: Bearer $token" $API_URL/apps/api/lineup/detail/\?lu_id=$series_id\&is_premium=true)

set -e

# check if token is empty
if [ -z "$token" ]; then
    echo "error token is empty"
    exit 1
fi

# check json res is empty
if [ -z "$json_response" ]; then
    echo "error json res is empty"
    exit 1
fi

# get episode_json
episode_json=$(echo $json_response | jq '.episodes')

#  get  detail_json
detail_json=$(echo $json_response | jq '.detail')

#  get total episode number
episode_count=$(echo $episode_json | jq 'length')

# get selected_episode
selected=${selected:-$((episode_count-1))}

# get selected_episode_json
selected_episode=$(echo $episode_json | jq ".[$selected]")

# print details
echo "series_id: $series_id"
echo "series_title: $(echo $detail_json | jq -r '.lu_title')"
echo ""

# create dir
mkdir -p data/$(echo $detail_json | jq -r '.lu_title')



selected_episodes=$(echo $episode_json | jq ".[$selected:$selection_end]")

# print selected
echo "已选集数:"
echo $selected_episodes | jq -r '.[].ep_title'

# download selected
# get id of selected ep ID
for ep_id in $(echo $selected_episodes | jq -r '.[].ep_id')
do
    # get title from ep json
    ep_title=$(echo $selected_episodes | jq -r ".[] | select(.ep_id == \"$ep_id\") | .ep_title")
    # replace 4011 to 4081
    ep_id=${ep_id/4011/4081}
    echo "downloading $ep_title"
    # get ep source
luid='5d40'
uuid='nqtwdys2-tl6w-41jz-yupj-5owfls7noxk1'
site_id='fodapp'
#token='eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJGT0QiLCJ1aWQiOiIiLCJkdl90eXBlIjoid2ViIiwiZHZfaWQiOiJNaWNyb3NvZnQgV2luZG93cyBOVCAxMC4wLjE3NzYzLjBfQ2hyb21lMTI2LjAifQ.urKCNm8tgJTiZjcajgIkr9V8Q47pXevI99MVfAtCIr8'
token='eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJGT0QiLCJ1aWQiOiIiLCJkdl90eXBlIjoid2ViIiwiZHZfaWQiOiJNaWNyb3NvZnQgV2luZG93cyBOVCAxMC4wLjE3NzYzLjBfQ2hyb21lMTI2LjAifQ.FBWSxy4LBQZdrTaeUSeelng2qpyJn0DbJz9gPQXJALg'
android_tv_user_agent='Mozilla/5.0 (Linux; Android 9; SHIELD Android TV) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.157 Safari/537.36'
cookie='ssid=ONmlkyk56FBwSG0+QR22Qeh7ZaRN6BBTvgNct6TZER0=; uuid=nqtwdys2-tl6w-41jz-yupj-5owfls7noxk1; CT=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJGT0QiLCJ1aWQiOiIiLCJkdl90eXBlIjoid2ViIiwiZHZfaWQiOiJNaWNyb3NvZnQgV2luZG93cyBOVCAxMC4wLjE3NzYzLjBfQ2hyb21lMTI2LjAifQ.urKCNm8tgJTiZjcajgIkr9V8Q47pXevI99MVfAtCIr8; UT=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJGT0QiLCJ1aWQiOiIiLCJkdl90eXBlIjoid2ViIiwiZHZfaWQiOiJNaWNyb3NvZnQgV2luZG93cyBOVCAxMC4wLjE3NzYzLjBfQ2hyb21lMTI2LjAifQ.FBWSxy4LBQZdrTaeUSeelng2qpyJn0DbJz9gPQXJALg; LC=0;'

# func:get web init
call_auth_viewing() {
    local epid=$ep_id
    local luid=$2
    local uuid=$3
    local site_id=$4
    local token=$5
    local cookie=$6

    local url="https://fod.fujitv.co.jp/apps/api/1/auth/contents/web?site_id=${site_id}&ep_id=${epid}&qa=high&uuid=${uuid}&starttime=0&is_pt=true"
    local response=$(curl -s -H "x-authorization: Bearer ${token}" -H "Cookie: ${cookie}" "${url}")

    echo "$response" | jq -r 'if .url then .url else .settings end'
}

# func get setting url
fetch_settings_url() {
    local settings_url=$1
    local response=$(curl -s -A "${android_tv_user_agent}" "${settings_url}")

    echo "$response" | jq -r '.video_selector[-1].url'
}

# func get m3u8 url
fetch_m3u8_url() {
    local m3u8_url=$1
    local response=$(curl -s -A "${android_tv_user_agent}" "${m3u8_url}")

    echo "$response"
}

# func get urls
generate_urls() {
    local m3u8_content=$1
    local formats=("${@:2}")
    local base_url=$(echo "$m3u8_content" | tail -n 1)
    local urls=()

    for fmt in "${formats[@]}"; do
        urls+=($(echo "$base_url" | sed "s/me[0-9]\{3\}/me${fmt}/"))
    done

    echo "${urls[@]}"
}

# main exec
m3u8_url=$(call_auth_viewing "$epid" "$luid" "$uuid" "$site_id" "$token" "$cookie")

if [[ $m3u8_url == *"tv_android"* ]]; then
    m3u8_url=$(fetch_settings_url "$m3u8_url")
fi

m3u8_data=$(fetch_m3u8_url "$m3u8_url")

formats=(113 115 117)
urls=($(generate_urls "$m3u8_data" "${formats[@]}"))

# print last url
echo "${urls[-1]}"

    # download ep
    streamlink hls://${urls[-1]} best --stream-segment-threads 10 --hls-live-edge 500 -o "data/$ep_title.ts"
    # convert ts to mp4
    export HOME=/www/wwwroot/www.chibimaru.cn/web/data
    echo "converting $ep_title"
    ffmpeg -hide_banner -loglevel quiet -stats -y -i "data/$ep_title.ts" -map v -c:v copy data/v.h264
    ffmpeg -hide_banner -loglevel quiet -stats -y -i "data/$ep_title.ts" -map a -c:a copy data/a.aac
    rm "data/$ep_title.ts"
    MP4Box -add data/a.aac -add data/v.h264 "data/$(echo $detail_json | jq -r '.lu_title')/$ep_title.mp4"
done
echo "download completed"

update: translated everything to english

The new episodes are still using m3u8