dvc: add: start having ERROR: unexpected error - 'Tqdm' object has no attribute 'desc' when upgrade from 2.10.2 to 2.11.0
Bug Report
Description
Running dvc add ./data
shows error ERROR: unexpected error - 'Tqdm' object has no attribute 'desc'
on version 2.11.0. In my case, one interesting thing is that if the data
folder is generated using dev data with the folder size ~55MB, both 2.10.2 and 2.11.0 work, but if I use prod data with the folder size ~2.59GB, only 2.10.2 works.
Reproduce
dvc remote default prod
dvc add ./data
Expected
2.11.0 should work on both dev and prod data like how 2.10.2 does. It should not depend on the data size.
Environment information
dvc doctor
DVC version: 2.11.0 (pip)
---------------------------------
Platform: Python 3.8.10 on Linux-5.10.104-linuxkit-x86_64-with-glibc2.29
Supports:
webhdfs (fsspec = 2022.5.0),
http (aiohttp = 3.8.1, aiohttp-retry = 2.4.6),
https (aiohttp = 3.8.1, aiohttp-retry = 2.4.6),
s3 (s3fs = 2022.5.0, boto3 = 1.21.21)
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 1
- Comments: 16 (13 by maintainers)
Commits related to this issue
- fix logging filenames - fixes https://github.com/iterative/dvc/issues/7896 - closes https://github.com/iterative/dvc-objects/pull/92 - closes https://github.com/tqdm/tqdm/pull/1341 — committed to iterative/dvc-data by casperdcl 2 years ago
- fix logging filenames - fixes https://github.com/iterative/dvc/issues/7896 - closes https://github.com/iterative/dvc-objects/pull/92 - closes https://github.com/tqdm/tqdm/pull/1341 — committed to iterative/dvc-data by casperdcl 2 years ago
OK, guys, I’ve found the issue! See this chunk of code in tqdm. When being run in a non-interactive environment (
isatty() is False
), the tqdm constructor hits this early return and doesn’t set thedesc
property.So, should I submit a patch to tqdm or should
dvc_objects
stop relying on existence of thedesc
attribute?Update I also opened https://github.com/iterative/dvc-objects/pull/92. Let’s solve this!
Ah srry for not seeing this earlier.
Thanks @sjawhar & @daavoo for the debugging efforts!
A few points:
main
anymore, so can’t be it…dvc-data
instead? I’ve opened https://github.com/iterative/dvc-data/pull/92 😃It’s about the size of the file, the line raising the error comes from:
https://github.com/iterative/dvc-objects/blob/b469fb63c4479c85f064c33696ef4f18ed92f65c/src/dvc_objects/hash.py#L133-L155
@casperdcl Maybe you have some idea here?