dvc: gdrive: raises unexpected error - name: drive version: v2 (again)

Bug Report

Description

As reported here, the latest version (2.11.0) of dvc on Ubuntu 22.04 LTS still has the bug reported in issue #5618. Downgrading dvc to 2.10.2 allows the push to work properly.

Reproduce

On an existing repository using gdrive, add some files then try to push them.

brett@br-workhorse:~/repos/arctic-ice$ dvc status
models.dvc:                                                                                                                                                                                                                                                                                           
        changed outs:
                modified:           models

brett@br-workhorse:~/repos/arctic-ice$ dvc add models
100% Adding...|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████|1/1 [00:01,  1.52s/file]

brett@br-workhorse:~/repos/arctic-ice$ dvc push                                                                                                                                                                                                                                                       
ERROR: unexpected error - name: drive  version: v2                                                                                                                                                                                                                                                    

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!

With extra verbosity:

brett@br-workhorse:~/repos/arctic-ice$ dvc push -vv
2022-06-29 19:59:18,216 TRACE: Namespace(all_branches=False, all_commits=False, all_tags=False, cd='.', cmd='push', cprofile=False, cprofile_dump=None, func=<class 'dvc.commands.data_sync.CmdDataPush'>, glob=False, instrument=False, instrument_open=False, jobs=None, parser=DvcParser(prog='dvc', usage=None, description='Data Version Control', formatter_class=<class 'argparse.RawTextHelpFormatter'>, conflict_handler='error', add_help=False), pdb=False, quiet=0, recursive=False, remote=None, run_cache=False, targets=[], verbose=2, version=None, viztracer=False, viztracer_depth=None, with_deps=False, yappi=False)
2022-06-29 19:59:18,399 TRACE:    16.49 ms in collecting stages from /home/brett/repos/arctic-ice
2022-06-29 19:59:18,405 TRACE:     6.02 ms in collecting stages from /home/brett/repos/arctic-ice/RadarMask
2022-06-29 19:59:18,406 TRACE:    18.37 mks in collecting stages from /home/brett/repos/arctic-ice/RadarMask/RadarMask
2022-06-29 19:59:18,420 DEBUG: Preparing to transfer data from '/home/brett/repos/arctic-ice/.dvc/cache' to '1uTdQZ6FntcI-JtX0ofC4LYPfFM37oZp0'
2022-06-29 19:59:18,420 DEBUG: Preparing to collect status from '1uTdQZ6FntcI-JtX0ofC4LYPfFM37oZp0'
2022-06-29 19:59:18,420 DEBUG: Collecting status from '1uTdQZ6FntcI-JtX0ofC4LYPfFM37oZp0'
2022-06-29 19:59:18,423 DEBUG: Querying 11 hashes via object_exists                                                                                                                                                                                                                                   
2022-06-29 19:59:18,635 DEBUG: GDrive remote auth with config '{'client_config_backend': 'settings', 'client_config_file': 'client_secrets.json', 'save_credentials': True, 'oauth_scope': ['https://www.googleapis.com/auth/drive', 'https://www.googleapis.com/auth/drive.appdata'], 'save_credentials_backend': 'file', 'save_credentials_file': '/home/brett/repos/arctic-ice/.dvc/tmp/gdrive-user-credentials.json', 'get_refresh_token': True, 'client_config': {'client_id': '710796635688-iivsgbgsb6uv1fap6635dhvuei09o66c.apps.googleusercontent.com', 'client_secret': 'a1Fz59uTpVNeG_VGuSKDLJXv', 'auth_uri': 'https://accounts.google.com/o/oauth2/auth', 'token_uri': 'https://oauth2.googleapis.com/token', 'revoke_uri': 'https://oauth2.googleapis.com/revoke', 'redirect_uri': ''}}'.
2022-06-29 19:59:18,637 DEBUG: GDrive remote auth with config '{'client_config_backend': 'settings', 'client_config_file': 'client_secrets.json', 'save_credentials': True, 'oauth_scope': ['https://www.googleapis.com/auth/drive', 'https://www.googleapis.com/auth/drive.appdata'], 'save_credentials_backend': 'file', 'save_credentials_file': '/home/brett/repos/arctic-ice/.dvc/tmp/gdrive-user-credentials.json', 'get_refresh_token': True, 'client_config': {'client_id': '710796635688-iivsgbgsb6uv1fap6635dhvuei09o66c.apps.googleusercontent.com', 'client_secret': 'a1Fz59uTpVNeG_VGuSKDLJXv', 'auth_uri': 'https://accounts.google.com/o/oauth2/auth', 'token_uri': 'https://oauth2.googleapis.com/token', 'revoke_uri': 'https://oauth2.googleapis.com/revoke', 'redirect_uri': ''}}'.
2022-06-29 19:59:18,638 DEBUG: GDrive remote auth with config '{'client_config_backend': 'settings', 'client_config_file': 'client_secrets.json', 'save_credentials': True, 'oauth_scope': ['https://www.googleapis.com/auth/drive', 'https://www.googleapis.com/auth/drive.appdata'], 'save_credentials_backend': 'file', 'save_credentials_file': '/home/brett/repos/arctic-ice/.dvc/tmp/gdrive-user-credentials.json', 'get_refresh_token': True, 'client_config': {'client_id': '710796635688-iivsgbgsb6uv1fap6635dhvuei09o66c.apps.googleusercontent.com', 'client_secret': 'a1Fz59uTpVNeG_VGuSKDLJXv', 'auth_uri': 'https://accounts.google.com/o/oauth2/auth', 'token_uri': 'https://oauth2.googleapis.com/token', 'revoke_uri': 'https://oauth2.googleapis.com/revoke', 'redirect_uri': ''}}'.
2022-06-29 19:59:18,639 DEBUG: GDrive remote auth with config '{'client_config_backend': 'settings', 'client_config_file': 'client_secrets.json', 'save_credentials': True, 'oauth_scope': ['https://www.googleapis.com/auth/drive', 'https://www.googleapis.com/auth/drive.appdata'], 'save_credentials_backend': 'file', 'save_credentials_file': '/home/brett/repos/arctic-ice/.dvc/tmp/gdrive-user-credentials.json', 'get_refresh_token': True, 'client_config': {'client_id': '710796635688-iivsgbgsb6uv1fap6635dhvuei09o66c.apps.googleusercontent.com', 'client_secret': 'a1Fz59uTpVNeG_VGuSKDLJXv', 'auth_uri': 'https://accounts.google.com/o/oauth2/auth', 'token_uri': 'https://oauth2.googleapis.com/token', 'revoke_uri': 'https://oauth2.googleapis.com/revoke', 'redirect_uri': ''}}'.
2022-06-29 19:59:18,641 DEBUG: GDrive remote auth with config '{'client_config_backend': 'settings', 'client_config_file': 'client_secrets.json', 'save_credentials': True, 'oauth_scope': ['https://www.googleapis.com/auth/drive', 'https://www.googleapis.com/auth/drive.appdata'], 'save_credentials_backend': 'file', 'save_credentials_file': '/home/brett/repos/arctic-ice/.dvc/tmp/gdrive-user-credentials.json', 'get_refresh_token': True, 'client_config': {'client_id': '710796635688-iivsgbgsb6uv1fap6635dhvuei09o66c.apps.googleusercontent.com', 'client_secret': 'a1Fz59uTpVNeG_VGuSKDLJXv', 'auth_uri': 'https://accounts.google.com/o/oauth2/auth', 'token_uri': 'https://oauth2.googleapis.com/token', 'revoke_uri': 'https://oauth2.googleapis.com/revoke', 'redirect_uri': ''}}'.
2022-06-29 19:59:18,642 DEBUG: GDrive remote auth with config '{'client_config_backend': 'settings', 'client_config_file': 'client_secrets.json', 'save_credentials': True, 'oauth_scope': ['https://www.googleapis.com/auth/drive', 'https://www.googleapis.com/auth/drive.appdata'], 'save_credentials_backend': 'file', 'save_credentials_file': '/home/brett/repos/arctic-ice/.dvc/tmp/gdrive-user-credentials.json', 'get_refresh_token': True, 'client_config': {'client_id': '710796635688-iivsgbgsb6uv1fap6635dhvuei09o66c.apps.googleusercontent.com', 'client_secret': 'a1Fz59uTpVNeG_VGuSKDLJXv', 'auth_uri': 'https://accounts.google.com/o/oauth2/auth', 'token_uri': 'https://oauth2.googleapis.com/token', 'revoke_uri': 'https://oauth2.googleapis.com/revoke', 'redirect_uri': ''}}'.
2022-06-29 19:59:18,642 DEBUG: GDrive remote auth with config '{'client_config_backend': 'settings', 'client_config_file': 'client_secrets.json', 'save_credentials': True, 'oauth_scope': ['https://www.googleapis.com/auth/drive', 'https://www.googleapis.com/auth/drive.appdata'], 'save_credentials_backend': 'file', 'save_credentials_file': '/home/brett/repos/arctic-ice/.dvc/tmp/gdrive-user-credentials.json', 'get_refresh_token': True, 'client_config': {'client_id': '710796635688-iivsgbgsb6uv1fap6635dhvuei09o66c.apps.googleusercontent.com', 'client_secret': 'a1Fz59uTpVNeG_VGuSKDLJXv', 'auth_uri': 'https://accounts.google.com/o/oauth2/auth', 'token_uri': 'https://oauth2.googleapis.com/token', 'revoke_uri': 'https://oauth2.googleapis.com/revoke', 'redirect_uri': ''}}'.
2022-06-29 19:59:18,643 DEBUG: GDrive remote auth with config '{'client_config_backend': 'settings', 'client_config_file': 'client_secrets.json', 'save_credentials': True, 'oauth_scope': ['https://www.googleapis.com/auth/drive', 'https://www.googleapis.com/auth/drive.appdata'], 'save_credentials_backend': 'file', 'save_credentials_file': '/home/brett/repos/arctic-ice/.dvc/tmp/gdrive-user-credentials.json', 'get_refresh_token': True, 'client_config': {'client_id': '710796635688-iivsgbgsb6uv1fap6635dhvuei09o66c.apps.googleusercontent.com', 'client_secret': 'a1Fz59uTpVNeG_VGuSKDLJXv', 'auth_uri': 'https://accounts.google.com/o/oauth2/auth', 'token_uri': 'https://oauth2.googleapis.com/token', 'revoke_uri': 'https://oauth2.googleapis.com/revoke', 'redirect_uri': ''}}'.
2022-06-29 19:59:18,644 DEBUG: GDrive remote auth with config '{'client_config_backend': 'settings', 'client_config_file': 'client_secrets.json', 'save_credentials': True, 'oauth_scope': ['https://www.googleapis.com/auth/drive', 'https://www.googleapis.com/auth/drive.appdata'], 'save_credentials_backend': 'file', 'save_credentials_file': '/home/brett/repos/arctic-ice/.dvc/tmp/gdrive-user-credentials.json', 'get_refresh_token': True, 'client_config': {'client_id': '710796635688-iivsgbgsb6uv1fap6635dhvuei09o66c.apps.googleusercontent.com', 'client_secret': 'a1Fz59uTpVNeG_VGuSKDLJXv', 'auth_uri': 'https://accounts.google.com/o/oauth2/auth', 'token_uri': 'https://oauth2.googleapis.com/token', 'revoke_uri': 'https://oauth2.googleapis.com/revoke', 'redirect_uri': ''}}'.
2022-06-29 19:59:18,645 DEBUG: GDrive remote auth with config '{'client_config_backend': 'settings', 'client_config_file': 'client_secrets.json', 'save_credentials': True, 'oauth_scope': ['https://www.googleapis.com/auth/drive', 'https://www.googleapis.com/auth/drive.appdata'], 'save_credentials_backend': 'file', 'save_credentials_file': '/home/brett/repos/arctic-ice/.dvc/tmp/gdrive-user-credentials.json', 'get_refresh_token': True, 'client_config': {'client_id': '710796635688-iivsgbgsb6uv1fap6635dhvuei09o66c.apps.googleusercontent.com', 'client_secret': 'a1Fz59uTpVNeG_VGuSKDLJXv', 'auth_uri': 'https://accounts.google.com/o/oauth2/auth', 'token_uri': 'https://oauth2.googleapis.com/token', 'revoke_uri': 'https://oauth2.googleapis.com/revoke', 'redirect_uri': ''}}'.
2022-06-29 19:59:18,645 DEBUG: GDrive remote auth with config '{'client_config_backend': 'settings', 'client_config_file': 'client_secrets.json', 'save_credentials': True, 'oauth_scope': ['https://www.googleapis.com/auth/drive', 'https://www.googleapis.com/auth/drive.appdata'], 'save_credentials_backend': 'file', 'save_credentials_file': '/home/brett/repos/arctic-ice/.dvc/tmp/gdrive-user-credentials.json', 'get_refresh_token': True, 'client_config': {'client_id': '710796635688-iivsgbgsb6uv1fap6635dhvuei09o66c.apps.googleusercontent.com', 'client_secret': 'a1Fz59uTpVNeG_VGuSKDLJXv', 'auth_uri': 'https://accounts.google.com/o/oauth2/auth', 'token_uri': 'https://oauth2.googleapis.com/token', 'revoke_uri': 'https://oauth2.googleapis.com/revoke', 'redirect_uri': ''}}'.
2022-06-29 19:59:18,647 ERROR: unexpected error - name: drive  version: v2                                                                                                                                                                                                                            
------------------------------------------------------------
Traceback (most recent call last):
  File "dvc/cli/__init__.py", line 185, in main
  File "dvc/cli/command.py", line 22, in do_run
  File "dvc/commands/data_sync.py", line 58, in run
  File "dvc/repo/__init__.py", line 49, in wrapper
  File "dvc/repo/push.py", line 68, in push
  File "dvc/data_cloud.py", line 109, in push
  File "dvc/data_cloud.py", line 88, in transfer
  File "dvc_data/transfer.py", line 159, in transfer
  File "dvc_data/status.py", line 179, in compare_status
  File "dvc_data/status.py", line 136, in status
  File "dvc_data/status.py", line 43, in _indexed_dir_hashes
  File "tqdm/std.py", line 1195, in __iter__
  File "dvc_objects/db.py", line 355, in list_hashes_exists
  File "concurrent/futures/_base.py", line 611, in result_iterator
  File "concurrent/futures/_base.py", line 439, in result
  File "concurrent/futures/_base.py", line 388, in __get_result
  File "concurrent/futures/thread.py", line 57, in run
  File "dvc_objects/fs/base.py", line 263, in exists
  File "pydrive2/fs/spec.py", line 225, in exists
  File "pydrive2/fs/spec.py", line 192, in _get_item_id
  File "pydrive2/fs/spec.py", line 174, in _path_to_item_ids
  File "pydrive2/fs/spec.py", line 170, in _get_cached_item_ids
  File "funcy/objects.py", line 50, in __get__
  File "funcy/objects.py", line 28, in __get__
  File "pydrive2/fs/spec.py", line 82, in _ids_cache
  File "pydrive2/fs/spec.py", line 132, in _gdrive_list
  File "funcy/objects.py", line 28, in __get__
  File "pydrive2/fs/spec.py", line 100, in _list_params
  File "funcy/decorators.py", line 45, in wrapper
  File "funcy/flow.py", line 127, in retry
  File "funcy/decorators.py", line 66, in __call__
  File "pydrive2/fs/spec.py", line 121, in _gdrive_shared_drive_id
  File "pydrive2/auth.py", line 64, in _decorated
  File "pydrive2/auth.py", line 671, in Authorize
  File "googleapiclient/_helpers.py", line 130, in positional_wrapper
  File "googleapiclient/discovery.py", line 287, in build
  File "googleapiclient/discovery.py", line 404, in _retrieve_discovery_doc
googleapiclient.errors.UnknownApiNameOrVersion: name: drive  version: v2
------------------------------------------------------------
2022-06-29 19:59:19,332 DEBUG: [Errno 95] no more link types left to try out: [Errno 95] Operation not supported
------------------------------------------------------------
Traceback (most recent call last):
  File "dvc/cli/__init__.py", line 185, in main
  File "dvc/cli/command.py", line 22, in do_run
  File "dvc/commands/data_sync.py", line 58, in run
  File "dvc/repo/__init__.py", line 49, in wrapper
  File "dvc/repo/push.py", line 68, in push
  File "dvc/data_cloud.py", line 109, in push
  File "dvc/data_cloud.py", line 88, in transfer
  File "dvc_data/transfer.py", line 159, in transfer
  File "dvc_data/status.py", line 179, in compare_status
  File "dvc_data/status.py", line 136, in status
  File "dvc_data/status.py", line 43, in _indexed_dir_hashes
  File "tqdm/std.py", line 1195, in __iter__
  File "dvc_objects/db.py", line 355, in list_hashes_exists
  File "concurrent/futures/_base.py", line 611, in result_iterator
  File "concurrent/futures/_base.py", line 439, in result
  File "concurrent/futures/_base.py", line 388, in __get_result
  File "concurrent/futures/thread.py", line 57, in run
  File "dvc_objects/fs/base.py", line 263, in exists
  File "pydrive2/fs/spec.py", line 225, in exists
  File "pydrive2/fs/spec.py", line 192, in _get_item_id
  File "pydrive2/fs/spec.py", line 174, in _path_to_item_ids
  File "pydrive2/fs/spec.py", line 170, in _get_cached_item_ids
  File "funcy/objects.py", line 50, in __get__
  File "funcy/objects.py", line 28, in __get__
  File "pydrive2/fs/spec.py", line 82, in _ids_cache
  File "pydrive2/fs/spec.py", line 132, in _gdrive_list
  File "funcy/objects.py", line 28, in __get__
  File "pydrive2/fs/spec.py", line 100, in _list_params
  File "funcy/decorators.py", line 45, in wrapper
  File "funcy/flow.py", line 127, in retry
  File "funcy/decorators.py", line 66, in __call__
  File "pydrive2/fs/spec.py", line 121, in _gdrive_shared_drive_id
  File "pydrive2/auth.py", line 64, in _decorated
  File "pydrive2/auth.py", line 671, in Authorize
  File "googleapiclient/_helpers.py", line 130, in positional_wrapper
  File "googleapiclient/discovery.py", line 287, in build
  File "googleapiclient/discovery.py", line 404, in _retrieve_discovery_doc
googleapiclient.errors.UnknownApiNameOrVersion: name: drive  version: v2

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "dvc_objects/fs/generic.py", line 68, in _try_links
  File "dvc_objects/fs/generic.py", line 28, in _link
  File "dvc_objects/fs/base.py", line 282, in reflink
  File "dvc_objects/fs/implementations/local.py", line 157, in reflink
  File "dvc_objects/fs/system.py", line 115, in reflink
  File "dvc_objects/fs/system.py", line 101, in _reflink_linux
OSError: [Errno 95] Operation not supported

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "dvc_objects/fs/generic.py", line 127, in _test_link
  File "dvc_objects/fs/generic.py", line 76, in _try_links
OSError: [Errno 95] no more link types left to try out
------------------------------------------------------------
2022-06-29 19:59:19,333 DEBUG: Removing '/home/brett/repos/.cFRhX4rL87pLC64LbFJkgt.tmp'
2022-06-29 19:59:19,333 DEBUG: Removing '/home/brett/repos/.cFRhX4rL87pLC64LbFJkgt.tmp'
2022-06-29 19:59:19,333 DEBUG: Removing '/home/brett/repos/.cFRhX4rL87pLC64LbFJkgt.tmp'
2022-06-29 19:59:19,333 DEBUG: Removing '/home/brett/repos/arctic-ice/.dvc/cache/.LAtLtAtBaEGib7LXwXJKVF.tmp'
2022-06-29 19:59:19,337 DEBUG: Version info for developers:
DVC version: 2.11.0 (deb)
---------------------------------
Platform: Python 3.8.3 on Linux-5.15.0-37-generic-x86_64-with-glibc2.14
Supports:
        azure (adlfs = 2022.4.0, knack = 0.9.0, azure-identity = 1.10.0),
        gdrive (pydrive2 = 1.10.1),
        gs (gcsfs = 2022.5.0),
        hdfs (fsspec = 2022.5.0, pyarrow = 8.0.0),
        webhdfs (fsspec = 2022.5.0),
        http (aiohttp = 3.8.1, aiohttp-retry = 2.4.8),
        https (aiohttp = 3.8.1, aiohttp-retry = 2.4.8),
        s3 (s3fs = 2022.5.0, boto3 = 1.21.21),
        ssh (sshfs = 2022.6.0),
        oss (ossfs = 2021.8.0),
        webdav (webdav4 = 0.9.7),
        webdavs (webdav4 = 0.9.7)
Cache types: hardlink, symlink
Cache directory: ext4 on /dev/nvme1n1p1
Caches: local
Remotes: gdrive
Workspace directory: ext4 on /dev/nvme1n1p1
Repo: dvc, git

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
2022-06-29 19:59:19,338 DEBUG: Analytics is enabled.
2022-06-29 19:59:19,339 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/tmp/tmp0h_rt9la']'
2022-06-29 19:59:19,341 DEBUG: Spawned '['daemon', '-q', 'analytics', '/tmp/tmp0h_rt9la']'

Expected

The push to succeed without errors.

Environment information

Output of dvc doctor:

brett@br-workhorse:~/repos/arctic-ice$ dvc doctor
DVC version: 2.11.0 (deb)
---------------------------------
Platform: Python 3.8.3 on Linux-5.15.0-37-generic-x86_64-with-glibc2.14
Supports:
        azure (adlfs = 2022.4.0, knack = 0.9.0, azure-identity = 1.10.0),
        gdrive (pydrive2 = 1.10.1),
        gs (gcsfs = 2022.5.0),
        hdfs (fsspec = 2022.5.0, pyarrow = 8.0.0),
        webhdfs (fsspec = 2022.5.0),
        http (aiohttp = 3.8.1, aiohttp-retry = 2.4.8),
        https (aiohttp = 3.8.1, aiohttp-retry = 2.4.8),
        s3 (s3fs = 2022.5.0, boto3 = 1.21.21),
        ssh (sshfs = 2022.6.0),
        oss (ossfs = 2021.8.0),
        webdav (webdav4 = 0.9.7),
        webdavs (webdav4 = 0.9.7)
Cache types: hardlink, symlink
Cache directory: ext4 on /dev/nvme1n1p1
Caches: local
Remotes: gdrive
Workspace directory: ext4 on /dev/nvme1n1p1
Repo: dvc, git

This is running with python 3.10.4 in a virtual environment set up with direnv.

brett@br-workhorse:~/repos/arctic-ice$ python --version
Python 3.10.4

brett@br-workhorse:~/repos/arctic-ice$ cat .envrc 
layout python-venv

brett@br-workhorse:~/repos/arctic-ice$ cat .direnv/python-venv-3.10.4/pyvenv.cfg 
home = /usr/bin
include-system-site-packages = false
version = 3.10.4

brett@br-workhorse:~/repos/arctic-ice$ which dvc
/usr/bin/dvc

brett@br-workhorse:~/repos/arctic-ice$ pip freeze -l
asttokens==2.0.5
backcall==0.2.0
certifi==2021.10.8
charset-normalizer==2.0.12
cycler==0.11.0
decorator==5.1.1
executing==0.8.3
fonttools==4.33.3
idna==3.3
imageio==2.18.0
imgaug==0.4.0
ipython==8.2.0
jedi==0.18.1
kiwisolver==1.4.2
matplotlib==3.5.1
matplotlib-inline==0.1.3
memory-profiler==0.60.0
networkx==2.8
numpy==1.22.3
opencv-contrib-python==4.5.5.64
opencv-python==4.5.5.64
packaging==21.3
parso==0.8.3
pexpect==4.8.0
pickleshare==0.7.5
Pillow==9.1.0
Pillow-SIMD==8.3.2.post0
prompt-toolkit==3.0.29
psutil==5.9.0
ptyprocess==0.7.0
pure-eval==0.2.2
Pygments==2.12.0
pyparsing==3.0.8
python-dateutil==2.8.2
PyWavelets==1.3.0
requests==2.27.1
scikit-image==0.19.2
scipy==1.8.0
Shapely==1.8.1.post1
six==1.16.0
stack-data==0.2.0
tifffile==2022.4.26
torch==1.11.0
torchvision==0.12.0
tqdm==4.64.0
traitlets==5.1.1
typing_extensions==4.2.0
urllib3==1.26.9
wcwidth==0.2.5

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 19 (10 by maintainers)

Commits related to this issue

Downgrade pyinstaller Per https://github.com/iterative/dvc/issues/7949#issuecomment-1170927010 — committed to iterative/dvc by daavoo 2 years ago
Downgrade pyinstaller Per https://github.com/iterative/dvc/issues/7949#issuecomment-1170927010 — committed to iterative/dvc by daavoo 2 years ago
pyinstaller: Restrict to 5.0 5.2 didn't fix https://github.com/iterative/dvc/issues/7949 — committed to iterative/dvc by daavoo 2 years ago
pyinstaller: Restrict to 5.0 5.2 didn't fix https://github.com/iterative/dvc/issues/7949 — committed to iterative/dvc by daavoo 2 years ago

Most upvoted comments

As I said, the last post is a setup that works. It is not using the key file. I added the key-file to the (private) repository just to see if that would work. The important step here is:

          pip install -U dvc[gdrive] pydrive2

sorenwacker on Jul 22, 2022

Seems like an issue with pyinstaller hook added in https://github.com/iterative/dvc/pull/5619

I the latest deb build the hook-googleapiclient.model.py didn’t work as expected, giving this warning:

38027 INFO: Loading module hook 'hook-googleapiclient.model.py' from '/dvc-s3-repo/dvc/scripts/pyinstaller/hooks'...
38070 WARNING: collect_data_files - skipping data collection for module 'googleapiclient.discovery' as it is not a package.

However, in the previous build (2.10.2) the hook worked as intended:

36358 INFO: Loading module hook 'hook-googleapiclient.model.py' from '/dvc-s3-repo/dvc/scripts/pyinstaller/hooks'...
36436 INFO: Loading module hook 'hook-dvc.utils.flatten.py' from '/dvc-s3-repo/dvc/scripts/pyinstaller/hooks'...

daavoo on Jun 30, 2022

this should be resolved by the linked pydrive patch once it is merged (and dvc-gdrive is updated)

pmrowla on Nov 15, 2022

Should note that we will probably continue to see the

WARNING: collect_data_files - skipping data collection for module 'googleapiclient.discovery' as it is not a package.

warning in the future, but it should no longer matter for us. The warning is generated due to an outdated pyinstaller.contrib hook for googleapiclient.model.

pmrowla on Nov 15, 2022

This appears to be broken again (w/pyinstaller 5.6)

from discord context: https://discord.com/channels/485586884165107732/485596304961962003/1041680983482695751

2022-11-14 11:45:50,298 ERROR: unexpected error - Failed to authenticate GDrive: name: drive  version: v2
------------------------------------------------------------
Traceback (most recent call last):
  File "pydrive2/fs/spec.py", line 70, in _wrap_errors
  File "pydrive2/fs/spec.py", line 150, in _service_auth
  File "pydrive2/auth.py", line 101, in _decorated
  File "pydrive2/auth.py", line 734, in Authorize
  File "googleapiclient/_helpers.py", line 130, in positional_wrapper
  File "googleapiclient/discovery.py", line 287, in build
  File "googleapiclient/discovery.py", line 404, in _retrieve_discovery_doc
googleapiclient.errors.UnknownApiNameOrVersion: name: drive  version: v2
...
pydrive2.fs.spec.GDriveAuthError: Failed to authenticate GDrive
------------------------------------------------------------
2022-11-14 11:45:51,251 DEBUG: Version info for developers:
DVC version: 2.34.0 (deb)
---------------------------------
Platform: Python 3.10.8 on Linux-5.15.0-1022-azure-x86_64-with-glibc2.31

We probably need explicit pyinstaller hooks for this library in either DVC or pydrive2 (or we need to contribute an actual fix upstream in pyinstaller’s hook)

.deb build logs for 2.34.1 https://github.com/iterative/dvc-s3-repo/actions/runs/3449002686/jobs/5756539961

71580 INFO: Loading module hook 'hook-googleapiclient.model.py' from '/root/.pyenv/versions/3.10.8/lib/python3.10/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
71635 WARNING: collect_data_files - skipping data collection for module 'googleapiclient.discovery' as it is not a package.

pmrowla on Nov 14, 2022

Bumping to pyinstaller 5.2. appears to haven’t fixed the issue. Hooks on the latest build still raise warnings and a user has reported the same bug in DVC version: 2.17.0 (deb):

https://github.com/iterative/dvc-s3-repo/runs/7703612801?check_suite_focus=true#step:6:2640

daavoo on Aug 9, 2022

It seems to be working now.

sorenwacker on Jul 22, 2022