TikTok-Api: [BUG] - Getting videos by Hashtag stopped working

Getting an exception:

Tag #viral, Error: Expecting value: line 1 column 1 (char 0)

It fails at this code:

with TikTokApi() as api:
    tag = api.hashtag(name="viral")

    print(tag.info())

    ->for video in tag.videos():
        print(video.id)

the for loop fires the exception and return 0 videos

This is the log output

urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): m.tiktok.com:443
urllib3.connectionpool - DEBUG - https://m.tiktok.com:443 "GET /api/challenge/detail" 200 308
urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): us.tiktok.com:443
urllib3.connectionpool - DEBUG - https://us.tiktok.com:443 "GET /api/challenge/item_list" 200 0
batch - ERROR - Tag #viral, Error: Expecting value: line 1 column 1 (char 0)

It was working perfectly on production, code above is a simplified example of my code, the issue happens with or without a proxy It was working up to the Nov 15th

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 28 (1 by maintainers)

Most upvoted comments

mandys on Nov 16, 2022

This works

def test_search(keyword, offset=0):
    params = {
        'keyword': keyword,
        'offset': offset,
    }
    cookies = {
        'ttwid': '** REDACTED **',
        'sessionid': '** REDACTED **',
    }
    request = requests.get("http://us.tiktok.com/api/search/item/full/", params=params, cookies=cookies)
    return(request.json())

print(test_search("#rstats"))

mesaleh on Nov 19, 2022

@mandys, i was try hit with full request params and response is blank… how you solve this issue?

azickri on Nov 18, 2022

For anyone still needing to get around this issue, I’m working on my own library that does work with getting videos under a tag. You don’t need to preload any cookies because it doesn’t handle API calls in the same way. I have yet to need to provide a proxy for bulk scraping. The methodology to get a large number of videos from a tag or user or comments under a video is handled a bit different, but this is explained in the documentation. I am actively working on the development of the library and am open to issues and contributions.

Russell-Newton on Dec 30, 2022

@azickri I am working with this, using as guideline the comments in others issues:

import json

#----------------------read cookies----------#
#get the cookies file downloaded in chrome with cookie manager
def get_cookies_from_file():
    with open('cookies.json') as f:
        cookies = json.load(f)

    cookies_kv = {}
   #Just use the name and the value properties
    for cookie in cookies:
        cookies_kv[cookie['name']] = cookie['value']
    return cookies_kv

#---------------------Request-----------------#
import requests

def test_search(keyword, offset=0):
   #Define params and headers
    params = {
        'keyword': keyword,
        'offset': offset
    }
   #Just test of headers, i think it's no neccesary
    headers = {
        "priority_regio":"CO",
        "tz_name=America":"%2FBogota"
    }
   #get all cookies
    cookies = get_cookies_from_file()

##just a test with only ttwid and session id
    # {
    #     'ttwid': <value of ttwid cookie>,
    #     'sessionid': <value of sessionid cookie>,
    # }

   #try request with /search/item/full
    request = requests.get("http://us.tiktok.com/api/search/item/full/", headers=headers, params=params, cookies=cookies)    
    
    #try request with /search/geneal/full
    # request = requests.get("http://us.tiktok.com/api/search/general/full/", headers=headers, params=params, cookies=cookies)

    #---print request---#
    # print(request.request.url)
    # print(request.request.body)
    # print(request.request.headers)

    return(request.json())

# Create an offset test, (tiktok return 12 videos in every run, I think that is neccesary read the propiertie has_more) 
offset = [0,12,24,36]
# run script for every offset

for count in offset:
    result = test_search(<word>, offset=count)
    # save the result in a file
    with open("hashtag.json", "a") as f:
        json.dump(result, f , indent=4)
        f.write('\n')
        f.close()

I am working with this code for search by keyword

matiquec on Nov 30, 2022

@DarinKumarnsit ,

I think, that not sessionid but csrf_session_id, you can see in developer tab Browser

azickri on Nov 21, 2022