dvc: dvc add no longer working
Bug Report
Issue name
dvc add <filename> causes an unexpected error:
Description
I’ve had DVC running on a Linux server for the past 3 months. Yesterday I set up a new conda environment with DVC 3.59.1 and now when I add data with “dvc add xxx” I get the following error:
“Adding… ERROR: unexpected error - no such column: “size” - should this be a string literal in single-quotes?”.
Let me add that I have tried to reset DVC (“dvc init”) and I have also deleted the .dvc directory and started from scratch, but the error persists. Maybe I made a mistake deleting the .dvc directory. Is there something else I need to do for a complete dvc reset?
Reproduce
- conda install dvc==3.57.0
- git init
- dvc init
- dvc add data/testfile.txt
Expected
Dvc adds file and does not show an error.
Environment information
Ubuntu 20.04 DVC 3.57.0
Output of dvc doctor:
$ dvc doctor
DVC version: 3.57.0 (conda)
Platform: Python 3.9.21 on Linux-6.8.0-52-generic-x86_64-with-glibc2.35 Subprojects: dvc_data = 3.16.9 dvc_objects = 5.1.0 dvc_render = 1.0.2 dvc_task = 0.40.2 scmrepo = 3.3.10 Supports: http (aiohttp = 3.11.12, aiohttp-retry = 2.8.3), https (aiohttp = 3.11.12, aiohttp-retry = 2.8.3) Config: Global: /home/uli/.config/dvc System: /etc/xdg/dvc Cache types: https://error.dvc.org/no-dvc-cache Caches: local Remotes: None Workspace directory: ext4 on /dev/nvme0n1 Repo: dvc, git Repo.site_cache_dir: /var/tmp/dvc/repo/1efc5571a372d9b6285ce7e59990bbae
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 8
- Comments: 20
Commits related to this issue
- MAINT: Fix sqlite regression by pin i.e. as seen here: https://github.com/iterative/dvc/issues/10692 — committed to HaoZeke/brms_idrot_repro by HaoZeke 2 years ago
Hi, I ran into the same problem today and found that it’s related to libsqlite version 3.49.1. In my case, running:
fixed the issue. Hope this helps!
Ok, I finally found the root cause: https://github.com/conda-forge/sqlite-feedstock/issues/130, which is a regression in a recent release of sqlite installed from conda. cc @sfinkens.
@skshetry: Thanks for jumping in. I looked at this in more detail last night. the crash happened in site-packages/dvc_data/hashfile/cache.py". The program was trying to do an insert into cache db and complained the column ‘size’ did not exist. The print for the added file looks as follows: (‘/test/data/test.txt’, ‘{“version”:1,“checksum”:“321037014704241475440851141032219623992”,“size”:5,“hash_info”:{“md5”:“6137cde4893c59f76f005a8123d8e8e6”}}’)]. So something’s off with that database and like @nobutoba has pointed out, it seems to be an issue with sqlite. His proposed fix worked for me.
@nobutoba : Thanks, you are a life saver!