icloud_photos_downloader: "Invalid Global Session" on every 100 items downloaded

Hi, I’m downloading a decent amount of photos accumulated for a few years, and the process seems to fail regularly with either “Invalid global session” on every 100 items, or randomly with “Gone (410)” errors.

It’s windows 10, freshly installed Python 3.6.3

If it matters – the connection is rather slow, I’m in Australia 😕

Log:

C:\Users\me\icloud_photos_downloader>download_photos.py d:\iCloud --username=me@gmail.com  --download-videos
Looking up all photos...
Downloading 5694 original photos and videos to d:\iCloud/ ...
Downloading d:\iCloud\2017/10/18\IMG_3671-original.JPG:   5%|????                                                                 | 300/5694 [59:04<15:28:58, 10.33s/it]Traceback (most recent call last):
  File "C:\Users\me\icloud_photos_downloader\download_photos.py", line 255, in <module>
    download()
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 697, in main
    rv = self.invoke(ctx)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "C:\Users\me\icloud_photos_downloader\download_photos.py", line 129, in download
    for photo in progress_bar:
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\tqdm\_tqdm.py", line 579, in __iter__
    for obj in iterable:
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyicloud\services\photos.py", line 281, in photos
    headers={'Content-type': 'text/plain'}
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\requests\sessions.py", line 555, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyicloud\base.py", line 100, in request
    self._raise_error(code, reason)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyicloud\base.py", line 122, in _raise_error
    raise api_error
pyicloud.exceptions.PyiCloudAPIResponseError: Invalid global session


C:\Users\me\icloud_photos_downloader>download_photos.py d:\iCloud --username=me@gmail.com --download-videos
Looking up all photos...
Downloading 5695 original photos and videos to d:\iCloud/ ...
Downloading d:\iCloud\2017/10/05\IMG_3486-original.MOV:   7%|?????                                                            | 377/5695 [1:00:54<304:35:47, 206.20s/it]Traceback (most recent call last):
  File "C:\Users\me\icloud_photos_downloader\download_photos.py", line 255, in <module>
    download()
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 697, in main
    rv = self.invoke(ctx)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "C:\Users\me\icloud_photos_downloader\download_photos.py", line 158, in download
    download_photo(photo, download_path, size, force_size, download_dir, progress_bar)
  File "C:\Users\me\icloud_photos_downloader\download_photos.py", line 230, in download_photo
    download_url = photo.download(size)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyicloud\services\photos.py", line 501, in download
    **kwargs
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\requests\sessions.py", line 521, in get
    return self.request('GET', url, **kwargs)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyicloud\base.py", line 74, in request
    self._raise_error(response.status_code, response.reason)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyicloud\base.py", line 122, in _raise_error
    raise api_error
pyicloud.exceptions.PyiCloudAPIResponseError: Gone (410)


C:\Users\me\icloud_photos_downloader>download_photos.py d:\iCloud --username=me@gmail.com --download-videos
Looking up all photos...
Downloading 5695 original photos and videos to d:\iCloud/ ...
Downloading d:\iCloud\2017/09/17\IMG_3256-original.JPG:   9%|??????                                                             | 500/5695 [1:18:19<54:56:44, 38.08s/it]Traceback (most recent call last):
  File "C:\Users\me\icloud_photos_downloader\download_photos.py", line 255, in <module>
    download()
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 697, in main
    rv = self.invoke(ctx)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "C:\Users\me\icloud_photos_downloader\download_photos.py", line 129, in download
    for photo in progress_bar:
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\tqdm\_tqdm.py", line 579, in __iter__
    for obj in iterable:
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyicloud\services\photos.py", line 281, in photos
    headers={'Content-type': 'text/plain'}
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\requests\sessions.py", line 555, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyicloud\base.py", line 100, in request
    self._raise_error(code, reason)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyicloud\base.py", line 122, in _raise_error
    raise api_error
pyicloud.exceptions.PyiCloudAPIResponseError: Invalid global session


C:\Users\me\icloud_photos_downloader>download_photos.py d:\iCloud --username=me@gmail.com --download-videos
Looking up all photos...
Downloading 5717 original photos and videos to d:\iCloud/ ...
Downloading d:\iCloud\2017/09/06\IMG_3070-original.MOV:  11%|????????                                                         | 631/5717 [1:36:58<749:26:03, 530.47s/it]Traceback (most recent call last):
  File "C:\Users\me\icloud_photos_downloader\download_photos.py", line 255, in <module>
    download()
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 697, in main
    rv = self.invoke(ctx)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "C:\Users\me\icloud_photos_downloader\download_photos.py", line 158, in download
    download_photo(photo, download_path, size, force_size, download_dir, progress_bar)
  File "C:\Users\me\icloud_photos_downloader\download_photos.py", line 230, in download_photo
    download_url = photo.download(size)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyicloud\services\photos.py", line 501, in download
    **kwargs
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\requests\sessions.py", line 521, in get
    return self.request('GET', url, **kwargs)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyicloud\base.py", line 74, in request
    self._raise_error(response.status_code, response.reason)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyicloud\base.py", line 122, in _raise_error
    raise api_error
pyicloud.exceptions.PyiCloudAPIResponseError: Gone (410)


C:\Users\me\icloud_photos_downloader>download_photos.py d:\iCloud --username=me@gmail.com --download-videos
Looking up all photos...
Downloading 5717 original photos and videos to d:\iCloud/ ...
Downloading d:\iCloud\2017/08/28\IMG_2939-original.JPG:  12%|?????????                                                             | 700/5717 [55:46<5:33:48,  3.99s/it]Traceback (most recent call last):
  File "C:\Users\me\icloud_photos_downloader\download_photos.py", line 255, in <module>
    download()
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 697, in main
    rv = self.invoke(ctx)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "C:\Users\me\icloud_photos_downloader\download_photos.py", line 129, in download
    for photo in progress_bar:
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\tqdm\_tqdm.py", line 579, in __iter__
    for obj in iterable:
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyicloud\services\photos.py", line 281, in photos
    headers={'Content-type': 'text/plain'}
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\requests\sessions.py", line 555, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyicloud\base.py", line 100, in request
    self._raise_error(code, reason)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyicloud\base.py", line 122, in _raise_error
    raise api_error
pyicloud.exceptions.PyiCloudAPIResponseError: Invalid global session


C:\Users\me\icloud_photos_downloader>download_photos.py d:\iCloud --username=me@gmail.com --download-videos
Looking up all photos...
Downloading 5717 original photos and videos to d:\iCloud/ ...
Downloading d:\iCloud\2017/08/14\IMG_2603-original.JPG:  14%|??????????                                                            | 800/5717 [44:53<5:13:56,  3.83s/it]Traceback (most recent call last):
  File "C:\Users\me\icloud_photos_downloader\download_photos.py", line 255, in <module>
    download()
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 697, in main
    rv = self.invoke(ctx)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "C:\Users\me\icloud_photos_downloader\download_photos.py", line 129, in download
    for photo in progress_bar:
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\tqdm\_tqdm.py", line 579, in __iter__
    for obj in iterable:
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyicloud\services\photos.py", line 281, in photos
    headers={'Content-type': 'text/plain'}
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\requests\sessions.py", line 555, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyicloud\base.py", line 100, in request
    self._raise_error(code, reason)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyicloud\base.py", line 122, in _raise_error
    raise api_error
pyicloud.exceptions.PyiCloudAPIResponseError: Invalid global session


C:\Users\me\icloud_photos_downloader>download_photos.py d:\iCloud --username=me@gmail.com --download-videos
Looking up all photos...
Downloading 5717 original photos and videos to d:\iCloud/ ...
Downloading d:\iCloud\2017/08/02\IMG_2411-original.JPG:  16%|???????????                                                          | 900/5717 [44:12<16:21:27, 12.23s/it]Traceback (most recent call last):
  File "C:\Users\me\icloud_photos_downloader\download_photos.py", line 255, in <module>
    download()
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 697, in main
    rv = self.invoke(ctx)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "C:\Users\me\icloud_photos_downloader\download_photos.py", line 129, in download
    for photo in progress_bar:
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\tqdm\_tqdm.py", line 579, in __iter__
    for obj in iterable:
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyicloud\services\photos.py", line 281, in photos
    headers={'Content-type': 'text/plain'}
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\requests\sessions.py", line 555, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyicloud\base.py", line 100, in request
    self._raise_error(code, reason)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyicloud\base.py", line 122, in _raise_error
    raise api_error
pyicloud.exceptions.PyiCloudAPIResponseError: Invalid global session


C:\Users\me\icloud_photos_downloader>download_photos.py d:\iCloud --username=me@gmail.com --download-videos
Looking up all photos...
Downloading 5717 original photos and videos to d:\iCloud/ ...
Downloading d:\iCloud\2017/07/23\IMG_2217-original.MOV:  17%|???????????                                                      | 965/5717 [1:02:29<415:16:48, 314.61s/it]Traceback (most recent call last):
  File "C:\Users\me\icloud_photos_downloader\download_photos.py", line 255, in <module>
    download()
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 697, in main
    rv = self.invoke(ctx)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\click\core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "C:\Users\me\icloud_photos_downloader\download_photos.py", line 158, in download
    download_photo(photo, download_path, size, force_size, download_dir, progress_bar)
  File "C:\Users\me\icloud_photos_downloader\download_photos.py", line 230, in download_photo
    download_url = photo.download(size)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyicloud\services\photos.py", line 501, in download
    **kwargs
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\requests\sessions.py", line 521, in get
    return self.request('GET', url, **kwargs)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyicloud\base.py", line 74, in request
    self._raise_error(response.status_code, response.reason)
  File "C:\Users\me\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyicloud\base.py", line 122, in _raise_error
    raise api_error
pyicloud.exceptions.PyiCloudAPIResponseError: Gone (410)

About this issue

Original URL
State: closed
Created 7 years ago
Comments: 16 (3 by maintainers)

Commits related to this issue

Re-authenticate on PyiCloudAPPIResponseError Sometimes when downloading many photos iCloud will return session errors. Catch these and re-authenticate. Fixes the "Invalid global session" Traceback. ... — committed to fnordahl/icloud_photos_downloader by fnordahl 6 years ago
Re-authenticate on PyiCloudAPIResponseError Sometimes when downloading many photos iCloud will return session errors. Catch these and re-authenticate. Fixes the "Invalid global session" Traceback. ... — committed to fnordahl/icloud_photos_downloader by fnordahl 6 years ago

Most upvoted comments

@ndbroadbent I’m still experiencing the same issue, with a failure after about 200 photos.

Traceback

 (most recent call last):
  File "./download_photos.py", line 304, in <module>
    download()
  File "/usr/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "./download_photos.py", line 142, in download
    for photo in progress_bar:
  File "/usr/lib/python2.7/site-packages/tqdm/_tqdm.py", line 579, in __iter__
    for obj in iterable:
  File "/usr/lib/python2.7/site-packages/pyicloud/services/photos.py", line 281, in photos
    headers={'Content-type': 'text/plain'}
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 555, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/lib/python2.7/site-packages/pyicloud/base.py", line 100, in request
    self._raise_error(code, reason)
  File "/usr/lib/python2.7/site-packages/pyicloud/base.py", line 122, in _raise_error
    raise api_error
pyicloud.exceptions.PyiCloudAPIResponseError: Invalid global session

FerasArabiat on Apr 26, 2018

Hi @dotcarls, sorry about that! I just updated the Docker image to use the latest version (1.4.0).

ndbroadbent on Dec 13, 2018

Hi everyone, sorry about this! I finally figured out the issue. The problem was that I didn’t handle session errors while fetching the next page of images, so it would sometimes crash during this request. But I’m now handling these errors as well, and retrying authentication.

Please update to version 1.3.4 which includes the fix: pip install icloudpd==1.3.4

ndbroadbent on Oct 10, 2018