dvc: dvc add no longer working

Bug Report

Issue name

dvc add <filename> causes an unexpected error:

Description

I’ve had DVC running on a Linux server for the past 3 months. Yesterday I set up a new conda environment with DVC 3.59.1 and now when I add data with “dvc add xxx” I get the following error:

“Adding… ERROR: unexpected error - no such column: “size” - should this be a string literal in single-quotes?”.

Let me add that I have tried to reset DVC (“dvc init”) and I have also deleted the .dvc directory and started from scratch, but the error persists. Maybe I made a mistake deleting the .dvc directory. Is there something else I need to do for a complete dvc reset?

Reproduce

  1. conda install dvc==3.57.0
  2. git init
  3. dvc init
  4. dvc add data/testfile.txt

Expected

Dvc adds file and does not show an error.

Environment information

Ubuntu 20.04 DVC 3.57.0

Output of dvc doctor:

$ dvc doctor

DVC version: 3.57.0 (conda)

Platform: Python 3.9.21 on Linux-6.8.0-52-generic-x86_64-with-glibc2.35 Subprojects: dvc_data = 3.16.9 dvc_objects = 5.1.0 dvc_render = 1.0.2 dvc_task = 0.40.2 scmrepo = 3.3.10 Supports: http (aiohttp = 3.11.12, aiohttp-retry = 2.8.3), https (aiohttp = 3.11.12, aiohttp-retry = 2.8.3) Config: Global: /home/uli/.config/dvc System: /etc/xdg/dvc Cache types: https://error.dvc.org/no-dvc-cache Caches: local Remotes: None Workspace directory: ext4 on /dev/nvme0n1 Repo: dvc, git Repo.site_cache_dir: /var/tmp/dvc/repo/1efc5571a372d9b6285ce7e59990bbae

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 8
  • Comments: 20

Commits related to this issue

Most upvoted comments

Hi, I ran into the same problem today and found that it’s related to libsqlite version 3.49.1. In my case, running:

conda install -c conda-forge libsqlite=3.48.0

fixed the issue. Hope this helps!

Ok, I finally found the root cause: https://github.com/conda-forge/sqlite-feedstock/issues/130, which is a regression in a recent release of sqlite installed from conda. cc @sfinkens.

@skshetry: Thanks for jumping in. I looked at this in more detail last night. the crash happened in site-packages/dvc_data/hashfile/cache.py". The program was trying to do an insert into cache db and complained the column ‘size’ did not exist. The print for the added file looks as follows: (‘/test/data/test.txt’, ‘{“version”:1,“checksum”:“321037014704241475440851141032219623992”,“size”:5,“hash_info”:{“md5”:“6137cde4893c59f76f005a8123d8e8e6”}}’)]. So something’s off with that database and like @nobutoba has pointed out, it seems to be an issue with sqlite. His proposed fix worked for me.

@nobutoba : Thanks, you are a life saver!