spack: "spack reindex" fails with "invalid ref count"

A previous install of git failed, leaving a half-installed git ghost. This is usually remedied via spack reindex, but this fails here.

My original command is:

$ spack install -v julia
==> Installing julia
==> git is already installed in /home/eschnett/src/spack/opt/spack/linux-centos7-x86_64/gcc-6.1.0-spack/git-2.9.3-kgah4b5gurqx2rswlt44xoinshfw7doi
Traceback (most recent call last):
  File "/home/eschnett/src/spack/bin/spack", line 192, in <module>
    main()
  File "/home/eschnett/src/spack/bin/spack", line 169, in main
    return_val = command(parser, args)
  File "/home/eschnett/src/spack/lib/spack/spack/cmd/install.py", line 92, in install
    explicit=True)
  File "/home/eschnett/src/spack/lib/spack/spack/package.py", line 919, in do_install
    run_tests=run_tests)
  File "/home/eschnett/src/spack/lib/spack/spack/package.py", line 1069, in do_install_dependencies
    dep.package.do_install(**kwargs)
  File "/home/eschnett/src/spack/lib/spack/spack/package.py", line 901, in do_install
    rec = spack.installed_db.get_record(self.spec)
  File "/home/eschnett/src/spack/lib/spack/spack/database.py", line 75, in converter
    return function(self, spec_like, *args, **kwargs)
  File "/home/eschnett/src/spack/lib/spack/spack/database.py", line 521, in get_record
    key = self._get_matching_spec_key(spec, **kwargs)
  File "/home/eschnett/src/spack/lib/spack/spack/database.py", line 516, in _get_matching_spec_key
    raise KeyError("No such spec in database! %s" % spec)
KeyError: 'No such spec in database! git@2.9.3%gcc@6.1.0-spack arch=linux-centos7-x86_64^autoconf@2.69%gcc@6.1.0-spack arch=linux-centos7-x86_64^bzip2@1.0.6%gcc@6.1.0-spack arch=linux-centos7-x86_64^curl@7.50.1%gcc@6.1.0-spack arch=linux-centos7-x86_64^expat@2.2.0%gcc@6.1.0-spack arch=linux-centos7-x86_64^gettext@0.19.8.1%gcc@6.1.0-spack+bzip2+curses+git~libunistring+libxml2+tar+xz arch=linux-centos7-x86_64^libsigsegv@2.10%gcc@6.1.0-spack arch=linux-centos7-x86_64^libxml2@2.9.4%gcc@6.1.0-spack~python arch=linux-centos7-x86_64^m4@1.4.17%gcc@6.1.0-spack+sigsegv arch=linux-centos7-x86_64^ncurses@6.0%gcc@6.1.0-spack arch=linux-centos7-x86_64^openssl@1.0.2h%gcc@6.1.0-spack arch=linux-centos7-x86_64^pcre@8.39%gcc@6.1.0-spack+utf arch=linux-centos7-x86_64^perl@5.24.0%gcc@6.1.0-spack+cpanm arch=linux-centos7-x86_64^tar@1.29%gcc@6.1.0-spack arch=linux-centos7-x86_64^xz@5.2.2%gcc@6.1.0-spack arch=linux-centos7-x86_64^zlib@1.2.8%gcc@6.1.0-spack arch=linux-centos7-x86_64'

To remedy, I try to reindex:

$ spack reindex
Traceback (most recent call last):
  File "/home/eschnett/src/spack/bin/spack", line 192, in <module>
    main()
  File "/home/eschnett/src/spack/bin/spack", line 169, in main
    return_val = command(parser, args)
  File "/home/eschnett/src/spack/lib/spack/spack/cmd/reindex.py", line 31, in reindex
    spack.installed_db.reindex(spack.install_layout)
  File "/home/eschnett/src/spack/lib/spack/spack/database.py", line 375, in reindex
    self._check_ref_counts()
  File "/home/eschnett/src/spack/lib/spack/spack/database.py", line 407, in _check_ref_counts
    (key, found, expected, self._index_path))
AssertionError: Invalid ref_count: 3f2b2nmz2brdouohnoxiik3zch7d4v7p: 31 (expected 30), in DB /home/eschnett/src/spack/opt/spack/.spack-db/index.yaml

About this issue

  • Original URL
  • State: closed
  • Created 10 years ago
  • Comments: 16 (14 by maintainers)

Commits related to this issue

Most upvoted comments

I think this should be a high-priority bug to fix: it messes up the entire repo, and we have no workaround; once a Spack installation gets into this state, we don’t know how to get it out. That prevents uninstalling packages (even after they’ve been uninstalled), and really breaks everything. We need, first and foremost, a way to get Spack out of this state and back into “clean” working order.

I think we need a procedure to rebuild this database from the installed packages in opt/spack. Unless we use an ACID database (we don’t), there will always be ways to break it.