conan: [bug] Conan is not able to download big packages

To help us debug your issue please explain:

Conan Version 1.12 and 1.18.2

According “Christian Charles” from Conan channel on Slack:

Hi all, I’m encountering a weird issue. With a big package (3.5GB), the download fails on my PC every time at the same size:

Downloading conan_sources.tgz
[===============                                   ] 1.07GB/3.56GB    ERROR: Download failed, check server, possibly try again
Transfer interrupted before complete: 1093543780 < 3556596602
Waiting 0 seconds to retry...
[===============                                   ] 1.07GB/3.56GB    ERROR: Download failed, check server, possibly try again
Transfer interrupted before complete: 1092076956 < 3556596602
Waiting 0 seconds to retry...
[===============                                   ] 1.07GB/3.56GB    ERROR: Download failed, check server, possibly try again
Transfer interrupted before complete: 1092712844 < 3556596602. [Remote: MYREMOTE]

More detailed log:

DEBUG :rest_client_common.py[136]: REST: get: http://artifactory.lyon.ts3d.lan/artifactory/api/conan/HolyGrail/v1/conans/ODA/2019/HE/stable/download_urls [2019-09-04 19:10:25,603]
DEBUG :uploader_downloader.py[209]: DOWNLOAD: http://artifactory.lyon.ts3d.lan/artifactory/api/conan/HolyGrail/v1/files/HE/ODA/2019/stable/export/conan_sources.tgz [2019-09-04 19:10:25,618]
Downloading conan_sources.tgz:  31%|###       | 1.02G/3.31G [01:33<03:50, 10.7MB/s]DEBUG :uploader_downloader.py[215]: <class 'conans.errors.ConanException'> [2019-09-04 19:11:59,454]
DEBUG :uploader_downloader.py[216]: Traceback (most recent call last):
                                      File "conan\conans\client\rest\uploader_downloader.py", line 210, in _download_file
                                      File "conan\conans\client\rest\uploader_downloader.py", line 286, in _download_data
                                    conans.errors.ConanException: Transfer interrupted before complete: 1093767356 < 3556596602
                                     [2019-09-04 19:11:59,454]
ERROR: Download failed, check server, possibly try again
Transfer interrupted before complete: 1093767356 < 3556596602
Waiting 0 seconds to retry...
DEBUG :uploader_downloader.py[209]: DOWNLOAD: http://artifactory.lyon.ts3d.lan/artifactory/api/conan/HolyGrail/v1/files/HE/ODA/2019/stable/export/conan_sources.tgz [2019-09-04 19:11:59,508]
Downloading conan_sources.tgz:  31%|###       | 1.02G/3.31G [01:50<03:50, 10.7MB/s]DEBUG :uploader_downloader.py[215]: <class 'conans.errors.ConanException'> [2019-09-04 19:13:31,887]
DEBUG :uploader_downloader.py[216]: Traceback (most recent call last):27, 11.9MB/s]
                                      File "conan\conans\client\rest\uploader_downloader.py", line 210, in _download_file
                                      File "conan\conans\client\rest\uploader_downloader.py", line 286, in _download_data
                                    conans.errors.ConanException: Transfer interrupted before complete: 1093793916 < 3556596602
                                     [2019-09-04 19:13:31,896]
ERROR: Download failed, check server, possibly try again
Transfer interrupted before complete: 1093793916 < 3556596602
Waiting 0 seconds to retry...
DEBUG :uploader_downloader.py[209]: DOWNLOAD: http://artifactory.lyon.ts3d.lan/artifactory/api/conan/HolyGrail/v1/files/HE/ODA/2019/stable/export/conan_sources.tgz [2019-09-04 19:13:31,910]
                                                                                   DEBUG :uploader_downloader.py[215]: <class 'conans.errors.ConanException'> [2019-09-04 19:15:04,181]
DEBUG :uploader_downloader.py[216]: Traceback (most recent call last):27, 11.9MB/s]
                                      File "conan\conans\client\rest\uploader_downloader.py", line 210, in _download_file
                                      File "conan\conans\client\rest\uploader_downloader.py", line 286, in _download_data
                                    conans.errors.ConanException: Transfer interrupted before complete: 1092720716 < 3556596602
                                     [2019-09-04 19:15:04,182]
ERROR: Download failed, check server, possibly try again
Transfer interrupted before complete: 1092720716 < 3556596602. [Remote: MYREMOTE]
Downloading conan_sources.tgz:  31%|###       | 1.02G/3.31G [04:38<10:27, 3.93MB/s]
Exception ignored in: <bound method tqdm.__del__ of Downloading conan_sources.tgz:  31%|###       | 1.02G/3.31G [03:04<03:27, 11.9MB/s]>
Traceback (most recent call last):%|###       | 1.02G/3.31G [01:32<03:28, 11.8MB/s]
  File "site-packages\tqdm\_tqdm.py", line 931, in __del__
  File "site-packages\tqdm\_tqdm.py", line 1133, in close
  File "site-packages\tqdm\_tqdm.py", line 496, in _decr_instances
  File "site-packages\tqdm\_monitor.py", line 52, in exit
  File "threading.py", line 1053, in join
RuntimeError: cannot join current thread

The server in this case is Artifactory CPP CE, not sure about the version.

Possible causes:

  • Artifactory limitation, the file to be download is 3.31.GB

  • tqdm bug or side-effect from connection interruption

  • I’ve read the CONTRIBUTING guide.

  • I’ve specified the Conan version, operating system version and any tool that can be relevant.

  • I’ve explained the steps to reproduce the error or the motivation/use case of the question/suggestion.

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 26 (19 by maintainers)

Most upvoted comments

So I re-install my python. My version is 3.7.4 : Python 3.7.4 (tags/v3.7.4:e09359112e, Jul 8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)] on win32 This win32 is weird since I use the installer python-3.7.4-amd64.exe provided https://www.python.org/ftp/python/3.7.4/python-3.7.4.exe

We are deploying this 3rdparty package on Windows and Linux. In my office I have 3 people having issue with the same error : failing to download it a 1GB/3.3GB both on Windows and Linux. On the other hand, I have 15 jenkins slaves having no issue and same for 15 developers. On my pc, I had 2 successes other 30+ trials.

I’ve been trying to reproduce an issue with both python 32-bit and python 64-bit, but no luck as well

https://airbrake.io/blog/python-exception-handling/memoryerror

“The memory for 32bits python is very low”. I have been reported that 64bits python have also some kind of issue with some dependency while installing Conan itself, this needs to be checked, please @danimtb or @uilianries have a look when possible.