pixivpy: "'NoneType' object is not iterable" for illust_ranking(**next_qs, req_auth=True)

Hi, I’m currently use this package for automatically download images from pixiv, down below is a part of my code:

downloaded = 0
logger.info("Starting ...")
stop_check = True
for entry in entries:
    next_qs = {"mode": f"{str(entry).replace('weekly_r18', 'week_r18').replace('daily_r18', 'day_r18')}"}
    i = 0
    while next_qs:
        try:
            if i > 3: break
            i = i + 1
            try:
                json_result = api.illust_ranking(**next_qs, req_auth=True)
            except Exception as e:
                logger.error(f"An error occurred while loading result for {entry}: {str(e)}")
                traceback.print_exc()
                time.sleep(5)
                refresh(PIXIV_REFRESH_TOKEN)
                continue
            for illust in json_result.illusts:
                if 'manga' in str(illust.tags): continue
                if illust.type == 'illust' and illust.page_count <= 15:
                    if str(illust.page_count) == '1':
                        try:
                            status = None
                            if 'r18' in entry:   status = api.download(path=f"art/r18/{str(entry).replace('_r18', '')}/", url=illust.meta_single_page.original_image_url, name=sanitize_file_name(f'{illust.id}---{illust.title}---{illust.user.id}---{illust.user.account}.jpg'))
                            if entry == 'day':   status = api.download(path=f"art/regular/daily/", url=illust.meta_single_page.original_image_url, name=sanitize_file_name(f'{illust.id}---{illust.title}---{illust.user.id}---{illust.user.account}.jpg'))
                            if entry == 'week':  status = api.download(path=f"art/regular/weekly/", url=illust.meta_single_page.original_image_url, name=sanitize_file_name(f'{illust.id}---{illust.title}---{illust.user.id}---{illust.user.account}.jpg'))
                            if entry == 'month': status = api.download(path=f"art/regular/monthly/", url=illust.meta_single_page.original_image_url, name=sanitize_file_name(f'{illust.id}---{illust.title}---{illust.user.id}---{illust.user.account}.jpg'))
                            # api.download(path="art/author_avatar", url=illust.user.profile_image_urls.medium, name=f'{illust.user.id}.jpg')
                            if status == True: 
                                logger.info(f"Successful downloaded image: {entry} {illust.id}")
                                downloaded += 1
                        except Exception as e:
                            traceback.print_exc()
                            logger.error(f"An error occurred in {entry} ({str(entry).replace('weekly_r18', 'week_r18').replace('daily_r18', 'day_r18')}, 1 page): " + str(e))
                            continue
                    else:
                        try:
                            download_path = ''
                            if 'r18' in entry:   
                                if illust.page_count > 3: continue
                                download_path = f"art/r18/{str(entry).replace('_r18', '')}/"
                            if entry == 'day':   download_path = "art/regular/daily/"
                            if entry == 'week':  download_path = "art/regular/weekly/"
                            if entry == 'month': download_path = "art/regular/monthly/"
                            counter = 1
                            for image in illust.meta_pages:
                                status = None
                                status = api.download(path=download_path, url=image.image_urls.original, name=sanitize_file_name(f'{illust.id}---{illust.title}---{illust.user.id}---{illust.user.account}---{counter}.jpg'))
                                if status == True: 
                                    logger.info(f"Successful downloaded image: {entry} {illust.id} | Page {counter}")
                                    downloaded += 1
                                counter += 1
                        except Exception as e:
                            logger.error(f"An error occurred in {entry} ({str(entry).replace('weekly_r18', 'week_r18').replace('daily_r18', 'day_r18')}, {illust.page_count} pages): " + str(e))
                            traceback.print_exc()
                            continue
            if json_result.next_url is not None:
                next_qs = api.parse_qs(json_result.next_url)
            else: break
        except Exception as e:
            logger.error(f"An error occurred while loading page for {entry}: {str(e)}")
            traceback.print_exc()
            time.sleep(5)
            refresh(PIXIV_REFRESH_TOKEN)
            continue
logger.info(f'Task completed download {downloaded} illustration.')

but these is a problem: Sometime, the illust_ranking(**next_qs, req_auth=True) return None, or someting, that my code cannot working with:

I am wondering, is that I’ve caused a rate-limit from pixiv, pr someing I’ve done wrong?

About this issue

Original URL
State: closed
Created 10 months ago
Comments: 15 (2 by maintainers)

Most upvoted comments

Thanks for your response! I will list everything i’ve changed in my code, to make sure that i didn’t make anything wrong:

Still call an auth instance with token in start:

api = AppPixivAPI()
api.auth(refresh_token=PIXIV_REFRESH_TOKEN)

Remove refresh() function, intsead making an api.auth() (no argument)

def main():
    global stop_check
    while True:
        try:
            if datetime.now().minute == 0 and datetime.now().hour == 3:
                stop_check = False
            if datetime.now().minute == 1 and datetime.now().hour == 3 and stop_check is False:
                stop_check = True
                downloaded = 0
                logger.info("Starting ...")
                api.auth()   # call the auth instance on start to make sure the token works
                json_result = api.illust_ranking(**next_qs, req_auth=True)   # req_auth arg can be removed cuz default is true
                # more stuff here, like what i've paste in the 1st comment
        except Exception as e:
            logger.error(f"An error occurred in main function: {str(e)}")
            time.sleep(5)
            api.auth()    # api.auth() intead of refresh(token) in every exception, im so lazy to do like your
        time.sleep(1)

I will try this out. If everything work fine, i will close this issue myself. Ortherwise, i will call you again 😄

tudubucket on Aug 26, 2023

In your refresh() I didn’t see you pass the refreshed access token back to the API instance, so it will probably still using the expired one. You can use api.auth() (without any argument) to trigger the refersh, instead of writting it by yourself.

TL;DR: Replace refresh(PIXIV_REFRESH_TOKEN) with api.auth().

Personally, I would record the expiration time of the current access token after each authentication (api.auth()). Then before each request is sent, check whether the access token is close to expiration (eg, within 2 minutes), and if so, refresh the access token first. This way I don’t need to wait until I encounter an error to refresh the access token.

Xdynix on Aug 26, 2023