dvc: unable to open database on mapped network drive

I just installed dvc. My repository is on a mapped network drive, and dvc cannot open the database file. It seems to work perfectly if the repository is on the local C: drive. for the example below, I just made a repositiory on my home network storage, which is mapped to the Z: drive.

timbp@Eucalypt MINGW64 /z/MyVideos
$ git init
Initialized empty Git repository in //192.168.29.3/MyLibrary/MyVideos/.git/

timbp@Eucalypt MINGW64 /z/MyVideos (master)
$ dvc init

You can now commit the changes to git.

+---------------------------------------------------------------------+
|                                                                     |
|        DVC has enabled anonymous aggregate usage analytics.         |
|     Read the analytics documentation (and how to opt-out) here:     |
|              https://dvc.org/doc/user-guide/analytics               |
|                                                                     |
+---------------------------------------------------------------------+

What's next?
------------
- Check out the documentation: https://dvc.org/doc
- Get help and share ideas: https://dvc.org/chat
- Star us on GitHub: https://github.com/iterative/dvc

timbp@Eucalypt MINGW64 /z/MyVideos (master)
$ git status
On branch master

No commits yet

Changes to be committed:
  (use "git rm --cached <file>..." to unstage)
        new file:   .dvc/.gitignore
        new file:   .dvc/config
        new file:   .dvc/plots/confusion.json
        new file:   .dvc/plots/default.json
        new file:   .dvc/plots/scatter.json
        new file:   .dvc/plots/smooth.json

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        chasings.mp4

timbp@Eucalypt MINGW64 /z/MyVideos (master)
$ git commit -m "testing"
[master (root-commit) 1f5f95a] testing
 6 files changed, 128 insertions(+)
 create mode 100644 .dvc/.gitignore
 create mode 100644 .dvc/config
 create mode 100644 .dvc/plots/confusion.json
 create mode 100644 .dvc/plots/default.json
 create mode 100644 .dvc/plots/scatter.json
 create mode 100644 .dvc/plots/smooth.json

timbp@Eucalypt MINGW64 /z/MyVideos (master)
$ dvc add chasings.mp4
ERROR: unexpected error - unable to open database file

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!

timbp@Eucalypt MINGW64 /z/MyVideos (master)
$ dvc status
ERROR: failed to obtain data status - unable to open database file

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!

timbp@Eucalypt MINGW64 /z/MyVideos (master)
$ dvc version
WARNING: Unable to detect supported link types, as cache directory '\\192.168.29.3\MyLibrary\MyVideos\.dvc\cache' doesn't exist. It is usually auto-created by commands such as `dvc add/fetch/pull/run/import`, but you could create it manually to enable this check.
DVC version: 1.5.0 (pip)
---------------------------------
Platform: Python 3.8.3 on Windows-10-10.0.19041-SP0
Supports: http, https
Repo: dvc, git

timbp@Eucalypt MINGW64 /z/MyVideos (master)
$ ls -l .dvc
total 0
-rw-r--r-- 1 timbp 197121 0 Aug 19 12:21 config
drwxr-xr-x 1 timbp 197121 0 Aug 19 12:21 plots/
drwxr-xr-x 1 timbp 197121 0 Aug 19 12:32 tmp/

timbp@Eucalypt MINGW64 /z/MyVideos (master)
$ ls -l .dvc/tmp/
total 2048
drwxr-xr-x 1 timbp 197121 0 Aug 19 12:21 index/
-rw-r--r-- 1 timbp 197121 7 Aug 19 12:33 lock
-rw-r--r-- 1 timbp 197121 7 Aug 19 12:33 updater.lock

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 20 (11 by maintainers)

Commits related to this issue

repo: allow user to specify different state and index tmp dir locations Allow user to override state's cache and remote's index files location through config when, say, the repository is located on a... — committed to agurtovoy/dvc by agurtovoy 3 years ago
repo: allow user to specify different state and index tmp dir locations Allow user to override state's cache and remote's index files location through config when, say, the repository is located on a... — committed to agurtovoy/dvc by agurtovoy 3 years ago
reincorporate PR 6111 into current master, re: #4420 (#6419) * rebased PR 6111 onto master * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci... — committed to iterative/dvc by nueschs 3 years ago
Revert "reincorporate PR 6111 into current master, re: #4420 (#6419)" This reverts commit b286c311f451261b16b63b6e3f33f88d09240136. — committed to efiop/dvc by efiop 3 years ago

Most upvoted comments

If anyone is looking for a workaround for this while the team is working on a more permanent fix, installing DVC from this fork will allow you to configure DVC to store its SQLite-based state and remote indexes somewhere else:

python -m pip install "git+https://github.com/agurtovoy/dvc.git#egg=dvc[s3]"

dvc config state.dir /tmp
dvc config index.dir /tmp

Here’s the corresponding PR: https://github.com/iterative/dvc/pull/6111

agurtovoy on Jun 4, 2021

This is also happening to me, when working with DvC inside a docker and my repository is mounted as a volume. Very easy to reproduce. I think it’s a sqlite problem more than DvC’s

ivsanro1 on Mar 3, 2021

dvc status --verbose
2021-02-01 15:40:34,009 DEBUG: Check for update is disabled.
2021-02-01 15:40:34,040 ERROR: unexpected error - unable to open database file
------------------------------------------------------------
Traceback (most recent call last):
 File "c:\program files\python38\lib\site-packages\dvc\main.py", line 90, in main
   ret = cmd.run()
 File "c:\program files\python38\lib\site-packages\dvc\command\status.py", line 49, in run
   st = self.repo.status(
 File "c:\program files\python38\lib\site-packages\dvc\repo\__init__.py", line 53, in wrapper
   with lock_repo(repo):
 File "c:\program files\python38\lib\contextlib.py", line 113, in __enter__
   return next(self.gen)
 File "c:\program files\python38\lib\site-packages\dvc\repo\__init__.py", line 41, in lock_repo
   with repo.lock, repo.state:
 File "c:\program files\python38\lib\site-packages\dvc\state.py", line 65, in __enter__
   self.load()
 File "c:\program files\python38\lib\site-packages\dvc\state.py", line 259, in load
   self.database = _connect_sqlite(self.state_file, {"nolock": 1})
 File "c:\program files\python38\lib\site-packages\dvc\state.py", line 527, in _connect_sqlite
   return sqlite3.connect(uri, uri=True)
sqlite3.OperationalError: unable to open database file
------------------------------------------------------------
2021-02-01 15:40:35,087 DEBUG: Version info for developers:
DVC version: 1.11.10 (pip)
---------------------------------
Platform: Python 3.8.6 on Windows-10-10.0.18362-SP0
Supports: All remotes
Cache types: <https://error.dvc.org/no-dvc-cache>
Caches: local
Remotes: local, ssh, ssh
Repo: dvc, git

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
2021-02-01 15:40:35,087 DEBUG: Analytics is enabled.
2021-02-01 15:40:35,446 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', 'C:\\Users\\AmisG01\\AppData\\Local\\Temp\\tmp18ye29i5']'
2021-02-01 15:40:35,524 DEBUG: Spawned '['daemon', '-q', 'analytics', 'C:\\Users\\AmisG01\\AppData\\Local\\Temp\\tmp18ye29i5']'

gamis on Feb 1, 2021

Fixed by https://github.com/iterative/dvc/pull/6419

efiop on Oct 8, 2021

@ivsanro1 Thanks! We are aware of this problem on nfs, cifs, overlayfs, etc and will be looking into it deeply in the coming weeks as 2.0 QA. Thank you for the feedback and stay tuned! 🙏

efiop on Mar 3, 2021

That’s correct. And I keep checking to see if there’s something wrong with the state file (not even created yet) or the permissions on the tmp directory or something. No leads until I search for this error message on github.

Apart from a more meaningful error message, I wonder if (at least for this situation), the sqlite database could be moved to %TEMP% instead, which is surely on a local drive.

gamis on Jan 28, 2021