Thanks @dtrifiro, this repo has always been private. I’ve been using it for several weeks already with no problem. It seems this error started appearing after I updated DVC to v2.10! After downgrading to v2.9.3 everything seems to work.

Here’s the whole debugging story FYI: Initially I used DVC v2.10.3. I tried running dvc get and dvc import with two git URLs: https://dagshub.com/aviv/data-repo, and using an access token (https://{user}:{token}@dagshub.com/aviv/data-repo). Both didn’t work. I’ve downgraded DVC to v2.10.1 and it looked much better, but still weird…

• First of all, dvc get and dvc import actually worked!
• They worked only when using the URL with the user:token prefix! Otherwise I got the same error as before (“No valid credentials provided”)
• After the successful dvc import, if I executed the same command again without any file changes, (almost) the same error appeared: “unexpected error - No valid credentials provided”.
• But if I push to dvc and git and then run dvc import, it worked…
• Finally, I downgraded DVC to v2.9.3 (the last version I remember worked 100%), and now running dvc import without any file changes results in “‘data.dvc’ didn’t change, skipping”, as it should’ve been. It works even with the standard HTTPS URL (without the user:token prefix), as it should.

Is the last value in the above configs credental.helper=store? If so, the reason for the failure might be a bad saved credential.

Can you check the following paths for any saved credentials and check whether they’re valid?

~/.git-credentials ~/.config/git/credentials

I’m not able to reproduce @avivio. Could you post the output of git config -l and maybe even the contents of the .dvc file you are trying to update (editing out any sensitive info)?

From our testing, http auth seems to be working. Let’s open a separate issue for ssh.

@avivio If you continue to have problems, please follow up with your auth setup 🙏 .

Hi @dtrifiro, I succeeded after trying the following. My original global git config

$ git config --global --list
user.email=xxx@gmail.com # Actually this is not the account of my private repository
core.editor="C:\Users\user\AppData\Local\Programs\Microsoft VS Code\bin\code" --wait
credential.http://xxx.xxx.xxx.xxx.provider=generic # This IP is the GitLab where my private repository is located

Setting global git credential.helper to manager-core doesn’t work

$ git config --global credential.helper manager-core
$ git config --global --list
user.email=xxx@gmail.com
core.editor="C:\Users\user\AppData\Local\Programs\Microsoft VS Code\bin\code" --wait
credential.http://xxx.xxx.xxx.xxx.provider=generic
credential.helper=manager-core
$ dvc import ...
Importing 'row_data (http://xxx.xxx.xxx.xxx/my-private-repo.git)' -> 'data'
ERROR: failed to import 'row_data' from 'http://xxx.xxx.xxx.xxx/my-private-repo.git'. - Failed to clone repo 'http://xxx.xxx.xxx.xxx/my-private-repo.git' to 'C:\Users\user\AppData\Local\Temp\tmp939z789ndvc-clone'

Setting global git credential.helper to store works!

$ git config --global credential.helper store
$ git config --global --list
user.email=xxx@gmail.com
core.editor="C:\Users\user\AppData\Local\Programs\Microsoft VS Code\bin\code" --wait
credential.http://xxx.xxx.xxx.xxx.provider=generic
credential.helper=store
$ dvc import ... # It works

In addition, the above three settings are all available for git pull.

My OS is Windows 11 and also has this problem. dvc doctor output:

DVC version: 2.34.0 (exe)
---------------------------------
Platform: Python 3.10.8 on Windows-10-10.0.22621-SP0
Subprojects:

Supports:
        azure (adlfs = 2022.10.0, knack = 0.10.0, azure-identity = 1.12.0),
        gdrive (pydrive2 = 1.14.0),
        gs (gcsfs = 2022.11.0),
        hdfs (fsspec = 2022.11.0, pyarrow = 10.0.0),
        http (aiohttp = 3.8.3, aiohttp-retry = 2.8.3),
        https (aiohttp = 3.8.3, aiohttp-retry = 2.8.3),
        oss (ossfs = 2021.8.0),
        s3 (s3fs = 2022.11.0, boto3 = 1.24.59),
        ssh (sshfs = 2022.6.0),
        webdav (webdav4 = 0.9.8),
        webdavs (webdav4 = 0.9.8),
        webhdfs (fsspec = 2022.11.0)
Cache types: <https://error.dvc.org/no-dvc-cache>
Caches: local
Remotes: s3
Workspace directory: NTFS on D:\
Repo: dvc, git

Try to dvc import data from my private repo:

$ git clone http://xxx.xxx.xxx.xxx/my-private-repo # sucess
$ cd my-private-repo
$ git config credential.helper manager-core
$ dvc import ... # import data from my my-private-repo with previus commit
Importing 'row_data (http://xxx.xxx.xxx.xxx/my-private-repo.git)' -> 'data'
ERROR: failed to import 'row_data' from 'http://xxx.xxx.xxx.xxx/my-private-repo.git'. - Failed to clone repo 'http://xxx.xxx.xxx.xxx/my-private-repo.git' to 'C:\Users\user\AppData\Local\Temp\tmpdlwm0mg3dvc-clone'

Retry with git config credential.helper store:

$ git clone http://xxx.xxx.xxx.xxx/my-private-repo # sucess
$ cd my-private-repo
$ git config credential.helper store
$ dvc import ... # import data from my-private-repo with previus commit
Importing 'row_data (http://xxx.xxx.xxx.xxx/my-private-repo.git)' -> 'data'
ERROR: failed to import 'row_data' from 'http://xxx.xxx.xxx.xxx/my-private-repo.git'. - Failed to clone repo 'http://xxx.xxx.xxx.xxx/my-private-repo.git' to 'C:\Users\user\AppData\Local\Temp\tmpr9sdc9r9dvc-clone'

...
2022-11-29 10:04:54,419 ERROR: failed to import 'row_data' from 'http://xx.xxx.xxx.xxx/my-private-repo.git'. - Failed to clone repo 'http://xx.xxx.xxx.xxx/my-private-repo.git' to 'C:\Users\user\AppData\Local\Temp\tmp2014udxmdvc-clone'
------------------------------------------------------------
Traceback (most recent call last):
  File "scmrepo\git\backend\dulwich\__init__.py", line 200, in clone
  File "dulwich\porcelain.py", line 551, in clone
  File "dulwich\client.py", line 760, in clone
  File "dulwich\client.py", line 837, in fetch
  File "dulwich\client.py", line 2076, in fetch_pack
  File "dulwich\client.py", line 1934, in _discover_references
  File "dulwich\client.py", line 2216, in _http_request
dulwich.client.HTTPUnauthorized: No valid credentials provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "dvc\scm.py", line 145, in clone
  File "scmrepo\git\__init__.py", line 143, in clone
  File "scmrepo\git\backend\dulwich\__init__.py", line 203, in clone
scmrepo.exceptions.CloneError: Failed to clone repo 'http://xxx.xxx.xxx.xxx/my-private-repo.git' to 'C:\Users\user\AppData\Local\Temp\tmp2014udxmdvc-clone'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "dvc\commands\imp.py", line 15, in run
  File "dvc\repo\imp.py", line 6, in imp
  File "dvc\repo\__init__.py", line 48, in wrapper
  File "dvc\repo\scm_context.py", line 156, in run
  File "dvc\repo\imp_url.py", line 98, in imp_url
  File "funcy\decorators.py", line 45, in wrapper
  File "dvc\stage\decorators.py", line 43, in rwlocked
  File "funcy\decorators.py", line 66, in __call__
  File "dvc\stage\__init__.py", line 543, in run
  File "funcy\decorators.py", line 45, in wrapper
  File "dvc\stage\decorators.py", line 43, in rwlocked
  File "funcy\decorators.py", line 66, in __call__
  File "dvc\stage\__init__.py", line 571, in _sync_import
  File "dvc\stage\imports.py", line 63, in sync_import
  File "dvc\dependency\repo.py", line 70, in download
  File "dvc\dependency\repo.py", line 99, in get_used_objs
  File "dvc\dependency\repo.py", line 113, in _get_used_and_obj
  File "contextlib.py", line 135, in __enter__
  File "dvc\external_repo.py", line 39, in external_repo
  File "dvc\external_repo.py", line 169, in _cached_clone
  File "funcy\decorators.py", line 45, in wrapper
  File "funcy\flow.py", line 274, in wrap_with
  File "funcy\decorators.py", line 66, in __call__
  File "dvc\external_repo.py", line 239, in _clone_default_branch
  File "dvc\scm.py", line 150, in clone
dvc.scm.CloneError: Failed to clone repo 'http://xxx.xxx.xxx.xxx/my-private-repo.git' to 'C:\Users\user\AppData\Local\Temp\tmp2014udxmdvc-clone'
------------------------------------------------------------
2022-11-29 10:04:54,423 DEBUG: Analytics is enabled.
2022-11-29 10:04:54,423 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', 'C:\\Users\\user\\AppData\\Local\\Temp\\tmpeaox6mf0']'
2022-11-29 10:04:54,427 DEBUG: Spawned '['daemon', '-q', 'analytics', 'C:\\Users\\user\\AppData\\Local\\Temp\\tmpeaox6mf0']'

@dberenbaum Thanks for hinting at the possible workaround. I’ve forwarded your comment to @dekromp who is using a Mac and got this error. We’ll report back. Good to know this is a priority.

@sisp Thanks for reporting. It’s hard to tell in which scenarios this breaks, but it’s clear at this point that it’s causing enough issues that we need to change the credential handling, and it’s been made a high priority.

In the meantime, can you try this workaround?

Om Mac I had to ssh-add -AK ~/.ssh/id_rsa to enable the agent and keychain: https://github.com/ronf/asyncssh/discussions/522

_Originally posted by @shcheklein in https://github.com/iterative/dvc/issues/7702#issuecomment-1304895509_

This is still happening for me after upgrading from version 2.9.5 to version 2.30.0

Environment information

Output of dvc doctor:

$ dvc doctor
DVC version: 2.30.0 (pip)
---------------------------------
Platform: Python 3.9.12 on macOS-12.6-x86_64-i386-64bit
Subprojects:
	dvc_data = 0.17.1
	dvc_objects = 0.7.0
	dvc_render = 0.0.12
	dvc_task = 0.1.3
	dvclive = 0.11.0
	scmrepo = 0.1.1
Supports:
	http (aiohttp = 3.8.3, aiohttp-retry = 2.8.3),
	https (aiohttp = 3.8.3, aiohttp-retry = 2.8.3)
Cache types: reflink, hardlink, symlink
Cache directory: apfs on /dev/disk1s1s1
Caches: local
Remotes: https
Workspace directory: apfs on /dev/disk1s1s1
Repo: dvc, git

Verbose output of dvc update:

$ dvc update data/dry.dvc -v
2022-10-19 12:17:55,342 DEBUG: Creating external repo https://dagshub.com/mana-bio/data-repo@None
2022-10-19 12:17:55,342 DEBUG: erepo: git clone 'https://dagshub.com/mana-bio/data-repo' to a temporary dir
2022-10-19 12:17:55,902 ERROR: failed update data - Failed to clone repo 'https://dagshub.com/mana-bio/data-repo' to '/var/folders/r2/qc7fk3l172d1wc1l6r1gyzcm0000gn/T/tmpla66ns6fdvc-clone'
------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/aviv/miniconda3/envs/playground/lib/python3.9/site-packages/scmrepo/git/backend/dulwich/__init__.py", line 200, in clone
    repo = clone_from()
  File "/Users/aviv/miniconda3/envs/playground/lib/python3.9/site-packages/dulwich/porcelain.py", line 538, in clone
    return client.clone(
  File "/Users/aviv/miniconda3/envs/playground/lib/python3.9/site-packages/dulwich/client.py", line 760, in clone
    result = self.fetch(path, target, progress=progress, depth=depth)
  File "/Users/aviv/miniconda3/envs/playground/lib/python3.9/site-packages/dulwich/client.py", line 837, in fetch
    result = self.fetch_pack(
  File "/Users/aviv/miniconda3/envs/playground/lib/python3.9/site-packages/dulwich/client.py", line 2075, in fetch_pack
    refs, server_capabilities, url = self._discover_references(
  File "/Users/aviv/miniconda3/envs/playground/lib/python3.9/site-packages/dulwich/client.py", line 1934, in _discover_references
    resp, read = self._http_request(url, headers)
  File "/Users/aviv/miniconda3/envs/playground/lib/python3.9/site-packages/dulwich/client.py", line 2215, in _http_request
    raise HTTPUnauthorized(resp.getheader("WWW-Authenticate"), url)
dulwich.client.HTTPUnauthorized: No valid credentials provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/aviv/miniconda3/envs/playground/lib/python3.9/site-packages/dvc/scm.py", line 145, in clone
    git = Git.clone(url, to_path, progress=pbar.update_git, **kwargs)
  File "/Users/aviv/miniconda3/envs/playground/lib/python3.9/site-packages/scmrepo/git/__init__.py", line 143, in clone
    backend.clone(url, to_path, **kwargs)
  File "/Users/aviv/miniconda3/envs/playground/lib/python3.9/site-packages/scmrepo/git/backend/dulwich/__init__.py", line 203, in clone
    raise CloneError(url, to_path) from exc
scmrepo.exceptions.CloneError: Failed to clone repo 'https://dagshub.com/mana-bio/data-repo' to '/var/folders/r2/qc7fk3l172d1wc1l6r1gyzcm0000gn/T/tmpla66ns6fdvc-clone'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/aviv/miniconda3/envs/playground/lib/python3.9/site-packages/dvc/commands/update.py", line 16, in run
    self.repo.update(
  File "/Users/aviv/miniconda3/envs/playground/lib/python3.9/site-packages/dvc/repo/__init__.py", line 49, in wrapper
    return f(repo, *args, **kwargs)
  File "/Users/aviv/miniconda3/envs/playground/lib/python3.9/site-packages/dvc/repo/update.py", line 40, in update
    stage.update(
  File "/Users/aviv/miniconda3/envs/playground/lib/python3.9/site-packages/dvc/stage/__init__.py", line 452, in update
    update_import(
  File "/Users/aviv/miniconda3/envs/playground/lib/python3.9/site-packages/dvc/stage/imports.py", line 23, in update_import
    stage.deps[0].update(rev=rev)
  File "/Users/aviv/miniconda3/envs/playground/lib/python3.9/site-packages/dvc/dependency/repo.py", line 87, in update
    with self._make_repo(locked=False) as repo:
  File "/Users/aviv/miniconda3/envs/playground/lib/python3.9/contextlib.py", line 119, in __enter__
    return next(self.gen)
  File "/Users/aviv/miniconda3/envs/playground/lib/python3.9/site-packages/dvc/external_repo.py", line 39, in external_repo
    path = _cached_clone(url, rev, for_write=for_write)
  File "/Users/aviv/miniconda3/envs/playground/lib/python3.9/site-packages/dvc/external_repo.py", line 169, in _cached_clone
    clone_path, shallow = _clone_default_branch(url, rev, for_write=for_write)
  File "/Users/aviv/miniconda3/envs/playground/lib/python3.9/site-packages/funcy/decorators.py", line 45, in wrapper
    return deco(call, *dargs, **dkwargs)
  File "/Users/aviv/miniconda3/envs/playground/lib/python3.9/site-packages/funcy/flow.py", line 274, in wrap_with
    return call()
  File "/Users/aviv/miniconda3/envs/playground/lib/python3.9/site-packages/funcy/decorators.py", line 66, in __call__
    return self._func(*self._args, **self._kwargs)
  File "/Users/aviv/miniconda3/envs/playground/lib/python3.9/site-packages/dvc/external_repo.py", line 239, in _clone_default_branch
    git = clone(url, clone_path)
  File "/Users/aviv/miniconda3/envs/playground/lib/python3.9/site-packages/dvc/scm.py", line 150, in clone
    raise CloneError(str(exc))
dvc.scm.CloneError: Failed to clone repo 'https://dagshub.com/mana-bio/data-repo' to '/var/folders/r2/qc7fk3l172d1wc1l6r1gyzcm0000gn/T/tmpla66ns6fdvc-clone'
------------------------------------------------------------
2022-10-19 12:17:55,915 DEBUG: Analytics is enabled.
2022-10-19 12:17:55,984 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/var/folders/r2/qc7fk3l172d1wc1l6r1gyzcm0000gn/T/tmphlvsybrd']'
2022-10-19 12:17:55,986 DEBUG: Spawned '['daemon', '-q', 'analytics', '/var/folders/r2/qc7fk3l172d1wc1l6r1gyzcm0000gn/T/tmphlvsybrd']'

This is fixed in 2.25.0

@guyrosin if you manually edit the .dvc file to include the token in the dependency url field I think dvc update will use the token. But otherwise you will need to wait for the linked fix to be merged & released

Still crashes with the same error using dvc v2.11.0 😦

❯ dvc get https://dagshub.com/aviv/data-repo README.md -v
2022-06-14 17:15:43,776 DEBUG: Creating external repo https://dagshub.com/aviv/data-repo@None
2022-06-14 17:15:43,776 DEBUG: erepo: git clone 'https://dagshub.com/aviv/data-repo' to a temporary dir
2022-06-14 17:15:44,238 DEBUG: Removing '/Users/guyrosin/code/playground/.eqznXKcQmyfXK5wC87zMvn'                                                                               
2022-06-14 17:15:44,238 ERROR: failed to get 'README.md' from 'https://dagshub.com/aviv/data-repo' - Failed to clone repo 'https://dagshub.com/aviv/data-repo' to '/var/folders/2x/3n66lzcd1gg06nk_sjjq319w0000gn/T/tmp_2h5e0wzdvc-clone'
------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/guyrosin/miniconda3/envs/rdkit-test/lib/python3.9/site-packages/scmrepo/git/backend/dulwich/__init__.py", line 196, in clone
    repo = clone_from()
  File "/Users/guyrosin/miniconda3/envs/rdkit-test/lib/python3.9/site-packages/dulwich/porcelain.py", line 443, in clone
    return client.clone(
  File "/Users/guyrosin/miniconda3/envs/rdkit-test/lib/python3.9/site-packages/dulwich/client.py", line 622, in clone
    result = self.fetch(path, target, progress=progress, depth=depth)
  File "/Users/guyrosin/miniconda3/envs/rdkit-test/lib/python3.9/site-packages/dulwich/client.py", line 699, in fetch
    result = self.fetch_pack(
  File "/Users/guyrosin/miniconda3/envs/rdkit-test/lib/python3.9/site-packages/dulwich/client.py", line 2075, in fetch_pack
    refs, server_capabilities, url = self._discover_references(
  File "/Users/guyrosin/miniconda3/envs/rdkit-test/lib/python3.9/site-packages/dulwich/client.py", line 1934, in _discover_references
    resp, read = self._http_request(url, headers, allow_compression=True)
  File "/Users/guyrosin/miniconda3/envs/rdkit-test/lib/python3.9/site-packages/dulwich/client.py", line 2218, in _http_request
    raise HTTPUnauthorized(resp.getheader("WWW-Authenticate"), url)
dulwich.client.HTTPUnauthorized: No valid credentials provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/guyrosin/miniconda3/envs/rdkit-test/lib/python3.9/site-packages/dvc/scm.py", line 145, in clone
    git = Git.clone(url, to_path, progress=pbar.update_git, **kwargs)
  File "/Users/guyrosin/miniconda3/envs/rdkit-test/lib/python3.9/site-packages/scmrepo/git/__init__.py", line 143, in clone
    backend.clone(url, to_path, **kwargs)
  File "/Users/guyrosin/miniconda3/envs/rdkit-test/lib/python3.9/site-packages/scmrepo/git/backend/dulwich/__init__.py", line 199, in clone
    raise CloneError(url, to_path) from exc
scmrepo.exceptions.CloneError: Failed to clone repo 'https://dagshub.com/aviv/data-repo' to '/var/folders/2x/3n66lzcd1gg06nk_sjjq319w0000gn/T/tmp_2h5e0wzdvc-clone'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/guyrosin/miniconda3/envs/rdkit-test/lib/python3.9/site-packages/dvc/commands/get.py", line 39, in _get_file_from_repo
    Repo.get(
  File "/Users/guyrosin/miniconda3/envs/rdkit-test/lib/python3.9/site-packages/dvc/repo/get.py", line 50, in get
    with external_repo(
  File "/Users/guyrosin/miniconda3/envs/rdkit-test/lib/python3.9/contextlib.py", line 119, in __enter__
    return next(self.gen)
  File "/Users/guyrosin/miniconda3/envs/rdkit-test/lib/python3.9/site-packages/dvc/external_repo.py", line 39, in external_repo
    path = _cached_clone(url, rev, for_write=for_write)
  File "/Users/guyrosin/miniconda3/envs/rdkit-test/lib/python3.9/site-packages/dvc/external_repo.py", line 169, in _cached_clone
    clone_path, shallow = _clone_default_branch(url, rev, for_write=for_write)
  File "/Users/guyrosin/miniconda3/envs/rdkit-test/lib/python3.9/site-packages/funcy/decorators.py", line 45, in wrapper
    return deco(call, *dargs, **dkwargs)
  File "/Users/guyrosin/miniconda3/envs/rdkit-test/lib/python3.9/site-packages/funcy/flow.py", line 274, in wrap_with
    return call()
  File "/Users/guyrosin/miniconda3/envs/rdkit-test/lib/python3.9/site-packages/funcy/decorators.py", line 66, in __call__
    return self._func(*self._args, **self._kwargs)
  File "/Users/guyrosin/miniconda3/envs/rdkit-test/lib/python3.9/site-packages/dvc/external_repo.py", line 239, in _clone_default_branch
    git = clone(url, clone_path)
  File "/Users/guyrosin/miniconda3/envs/rdkit-test/lib/python3.9/site-packages/dvc/scm.py", line 150, in clone
    raise CloneError(str(exc))
dvc.scm.CloneError: Failed to clone repo 'https://dagshub.com/aviv/data-repo' to '/var/folders/2x/3n66lzcd1gg06nk_sjjq319w0000gn/T/tmp_2h5e0wzdvc-clone'
------------------------------------------------------------
2022-06-14 17:15:44,258 DEBUG: Analytics is enabled.
2022-06-14 17:15:44,308 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/var/folders/2x/3n66lzcd1gg06nk_sjjq319w0000gn/T/tmpkmarbkqi']'
2022-06-14 17:15:44,310 DEBUG: Spawned '['daemon', '-q', 'analytics', '/var/folders/2x/3n66lzcd1gg06nk_sjjq319w0000gn/T/tmpkmarbkqi']'

Output of dvc doctor:

$ dvc doctor
DVC version: 2.11.0 (pip)
---------------------------------
Platform: Python 3.9.12 on macOS-12.3.1-x86_64-i386-64bit
Supports:
        gdrive (pydrive2 = 1.10.1),
        hdfs (fsspec = 2022.5.0, pyarrow = 8.0.0),
        webhdfs (fsspec = 2022.5.0),
        http (aiohttp = 3.8.1, aiohttp-retry = 2.4.6),
        https (aiohttp = 3.8.1, aiohttp-retry = 2.4.6)
Cache types: reflink, hardlink, symlink
Cache directory: apfs on /dev/disk1s1s1
Caches: local
Remotes: https
Workspace directory: apfs on /dev/disk1s1s1
Repo: dvc, git

Can confirm this is happening, starting from https://github.com/iterative/dvc/pull/7554