dropbox-sdk-python: ReadTimeout

There is an error while retrieving list of files from folder with almost 100k files. But all files were retrieved successfully.

Traceback (most recent call last):
  File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/urllib3/connectionpool.py", line 386, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/urllib3/connectionpool.py", line 382, in _make_request
    httplib_response = conn.getresponse()
  File ".pyenv/versions/3.6.4/lib/python3.6/http/client.py", line 1331, in getresponse
    response.begin()
  File ".pyenv/versions/3.6.4/lib/python3.6/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File ".pyenv/versions/3.6.4/lib/python3.6/http/client.py", line 258, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File ".pyenv/versions/3.6.4/lib/python3.6/socket.py", line 586, in readinto
    return self._sock.recv_into(b)
  File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py", line 291, in recv_into
    raise timeout('The read operation timed out')
socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/requests/adapters.py", line 445, in send
    timeout=timeout
  File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/urllib3/connectionpool.py", line 649, in urlopen
    _stacktrace=sys.exc_info()[2])
  File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/urllib3/util/retry.py", line 357, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise
    raise value
  File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/urllib3/connectionpool.py", line 388, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/urllib3/connectionpool.py", line 308, in _raise_timeout
    raise ReadTimeoutError(self, url, "Read timed out. (read timeout=%s)" % timeout_value)
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='api.dropboxapi.com', port=443): Read timed out. (read timeout=30)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "apps/creatives_manager/processors/base.py", line 144, in _get_files_from_dropbox
    result = self.dropbox.files_list_folder_continue(result.cursor)
  File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/dropbox/base.py", line 1583, in files_list_folder_continue
    None,
  File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/dropbox/dropbox.py", line 274, in request
    timeout=timeout)
  File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/dropbox/dropbox.py", line 365, in request_json_string_with_retry
    timeout=timeout)
  File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/dropbox/dropbox.py", line 449, in request_json_string
    timeout=timeout,
  File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/requests/sessions.py", line 559, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/requests/sessions.py", line 512, in request
    resp = self.send(prep, **send_kwargs)
  File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/requests/sessions.py", line 622, in send
    r = adapter.send(request, **kwargs)
  File ".pyenv/versions/marketing-ua-reports/lib/python3.6/site-packages/requests/adapters.py", line 526, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='api.dropboxapi.com', port=443): Read timed out. (read timeout=30)

Repeating of call self.dropbox.files_list_folder_continue(result.cursor) and enlarging timeout don’t help.

About this issue

  • Original URL
  • State: open
  • Created 6 years ago
  • Comments: 15 (7 by maintainers)

Most upvoted comments

Thanks for trying that and following up. It sounds like this is due to the API call taking a long time due to the large number of files then.

Please try using a smaller limit, when first calling files_list_folder. The default is effectively in the thousands, so please try something significantly smaller, such as 50.