spack: Spack Bug : Adding Explicit Dependency to Package Can Make it Unusable by Dependents
Since Spack currently always uses the default version of a dependency during spec normalization, I often add explicit dependencies to packages on install to prevent unnecessary package installs. For example, if I need to install a^b+debug and c^b, I’ll typically install a^b+debug and c^b+debug to prevent an unnecessary install of b~debug. Unfortunately, I’ve found that doing this can make packages unusable as dependencies by other packages.
In the error case that I was able to find, I wanted to install the following set of packages in an empty Spack repository:
qt@4.8.6+webkitvtk@6.1.0~opengl2, which depends onqtvisit@2.10.2, which depends onqt@4.8.6andvtk@6.1.0~opengl2
In order to prevent redundant installs, I attempted to install each of the Qt-dependent packages with an explicit ^qt@4.8.6+webkit dependency in the order listed above. This caused the visit@2.10.2 install to attempt to install a new instance of vtk@6.1.0~opengl2 that required new instances of the libxcb and qt packages. I haven’t allowed this new vtk@6.1.0~opengl2 to fully install yet, but I found that the libxcb installed in this way has a different hash than the libxcb installed through qt@4.8.6+webkit but is otherwise identical (even querying either one of the hashes with spack find yields both installs).
I also tried to install visit@2.10.2 ^"/pidklxj" ^qt@4.8.6+webkit (where /pidklxj is the hash for the vtk@6.1.0~opengl2 ^qt@4.8.6+webkit install), but encountered the following error:
==> Error: Specs
vtk@6.1.0%gcc@4.4.7~opengl2 arch=linux-redhat6-x86_64
^cmake@3.6.0%gcc@4.4.7~doc+ncurses+openssl~qt arch=linux-redhat6-x86_64
^ncurses@6.0%gcc@4.4.7 arch=linux-redhat6-x86_64
^openssl@1.0.2h%gcc@4.4.7 arch=linux-redhat6-x86_64
^zlib@1.2.8%gcc@4.4.7 arch=linux-redhat6-x86_64
^qt@4.8.6%gcc@4.4.7~gtk~krellpatch~mesa+webkit arch=linux-redhat6-x86_64
^bison@3.0.4%gcc@4.4.7 arch=linux-redhat6-x86_64
^m4@1.4.17%gcc@4.4.7+sigsegv arch=linux-redhat6-x86_64
^libsigsegv@2.10%gcc@4.4.7 arch=linux-redhat6-x86_64
^dbus@1.11.2%gcc@4.4.7 arch=linux-redhat6-x86_64
^expat@2.1.0%gcc@4.4.7 arch=linux-redhat6-x86_64
^flex@2.6.0%gcc@4.4.7 arch=linux-redhat6-x86_64
^fontconfig@2.11.1%gcc@4.4.7 arch=linux-redhat6-x86_64
^freetype@2.5.3%gcc@4.4.7 arch=linux-redhat6-x86_64
^libpng@1.6.16%gcc@4.4.7 arch=linux-redhat6-x86_64
^libxml2@2.9.2%gcc@4.4.7~python arch=linux-redhat6-x86_64
^xz@5.2.2%gcc@4.4.7 arch=linux-redhat6-x86_64
^glib@2.49.4%gcc@4.4.7 arch=linux-redhat6-x86_64
^gettext@0.19.8.1%gcc@4.4.7+bzip2+curses+git~libunistring+libxml2+tar+xz arch=linux-redhat6-x86_64
^bzip2@1.0.6%gcc@4.4.7 arch=linux-redhat6-x86_64
^tar@1.29%gcc@4.4.7 arch=linux-redhat6-x86_64
^libffi@3.2.1%gcc@4.4.7 arch=linux-redhat6-x86_64
^pcre@8.38%gcc@4.4.7+utf arch=linux-redhat6-x86_64
^pkg-config@0.29.1%gcc@4.4.7 arch=linux-redhat6-x86_64
^gperf@3.0.4%gcc@4.4.7 arch=linux-redhat6-x86_64
^icu4c@54_1%gcc@4.4.7 arch=linux-redhat6-x86_64
^jpeg@9b%gcc@4.4.7 arch=linux-redhat6-x86_64
^libmng@2.0.2%gcc@4.4.7 arch=linux-redhat6-x86_64
^lcms@2.6%gcc@4.4.7 arch=linux-redhat6-x86_64
^libtiff@4.0.3%gcc@4.4.7 arch=linux-redhat6-x86_64
^libxcb@1.11.1%gcc@4.4.7 arch=linux-redhat6-x86_64
^libpthread-stubs@0.3%gcc@4.4.7 arch=linux-redhat6-x86_64
^libxau@1.0.8%gcc@4.4.7 arch=linux-redhat6-x86_64
^xproto@7.0.29%gcc@4.4.7 arch=linux-redhat6-x86_64
^xorg-util-macros@1.19.0%gcc@4.4.7 arch=linux-redhat6-x86_64
^python@2.7.12%gcc@4.4.7~tk~ucs4 arch=linux-redhat6-x86_64
^readline@6.3%gcc@4.4.7 arch=linux-redhat6-x86_64
^sqlite@3.8.5%gcc@4.4.7 arch=linux-redhat6-x86_64
^xcb-proto@1.11%gcc@4.4.7 arch=linux-redhat6-x86_64
and
vtk@6.1.0%gcc@4.4.7~opengl2 arch=linux-redhat6-x86_64
^qt@4.8.6%gcc@4.4.7~gtk~krellpatch~mesa+webkit arch=linux-redhat6-x86_64
^dbus@1.11.2%gcc@4.4.7 arch=linux-redhat6-x86_64
^expat@2.1.0%gcc@4.4.7 arch=linux-redhat6-x86_64
^fontconfig@2.11.1%gcc@4.4.7 arch=linux-redhat6-x86_64
^freetype@2.5.3%gcc@4.4.7 arch=linux-redhat6-x86_64
^libpng@1.6.16%gcc@4.4.7 arch=linux-redhat6-x86_64
^zlib@1.2.8%gcc@4.4.7 arch=linux-redhat6-x86_64
^libxml2@2.9.2%gcc@4.4.7~python arch=linux-redhat6-x86_64
^xz@5.2.2%gcc@4.4.7 arch=linux-redhat6-x86_64
^glib@2.49.4%gcc@4.4.7 arch=linux-redhat6-x86_64
^gettext@0.19.8.1%gcc@4.4.7+bzip2+curses+git~libunistring+libxml2+tar+xz arch=linux-redhat6-x86_64
^bzip2@1.0.6%gcc@4.4.7 arch=linux-redhat6-x86_64
^ncurses@6.0%gcc@4.4.7 arch=linux-redhat6-x86_64
^tar@1.29%gcc@4.4.7 arch=linux-redhat6-x86_64
^libffi@3.2.1%gcc@4.4.7 arch=linux-redhat6-x86_64
^pcre@8.38%gcc@4.4.7+utf arch=linux-redhat6-x86_64
^gperf@3.0.4%gcc@4.4.7 arch=linux-redhat6-x86_64
^icu4c@54_1%gcc@4.4.7 arch=linux-redhat6-x86_64
^jpeg@9b%gcc@4.4.7 arch=linux-redhat6-x86_64
^libmng@2.0.2%gcc@4.4.7 arch=linux-redhat6-x86_64
^lcms@2.6%gcc@4.4.7 arch=linux-redhat6-x86_64
^libtiff@4.0.3%gcc@4.4.7 arch=linux-redhat6-x86_64
^libxcb@1.11.1%gcc@4.4.7 arch=linux-redhat6-x86_64
^libpthread-stubs@0.3%gcc@4.4.7 arch=linux-redhat6-x86_64
^libxau@1.0.8%gcc@4.4.7 arch=linux-redhat6-x86_64
^xproto@7.0.29%gcc@4.4.7 arch=linux-redhat6-x86_64
^xorg-util-macros@1.19.0%gcc@4.4.7 arch=linux-redhat6-x86_64
^xcb-proto@1.11%gcc@4.4.7 arch=linux-redhat6-x86_64
^openssl@1.0.2h%gcc@4.4.7 arch=linux-redhat6-x86_64
have the same SHA-1 prefix!
It’s apparent in this output that the installs are pretty much identical with the only real difference being the libxcb dependency, which isn’t as fully filled out in the second case since python is included earlier in the visit@2.10.2 dependency list.
@tgamblin, @becker33: Is this difference in how the specification is being filled out causing these problems? If not, what else do you suppose could be causing this problem? Thanks in advance to all that provide input.
About this issue
- Original URL
- State: closed
- Created 10 years ago
- Comments: 15 (15 by maintainers)
could be related to https://github.com/LLNL/spack/issues/1457#issuecomment-239318984 or https://github.com/LLNL/spack/pull/1409
maybe a more general question and suggested solution: when there is a DAG with a package that is used differently (
buildvsbuild+link), why do we actually try to (quote @becker33)? So if
libxcb [build] -> pythonthen should not it stay like that with the same hash irrespectively oflibxcbbeing build alone or as dependency of another package likevisit [build,link] -> python? My understanding is that the waylibxcbis build using python won’t depend on the same version of python being used elsewhere in DAG but as a different type of dependency (linkorrun). Therebylibxcbshould have the samehashin these two cases.@becker33 The fact that Spack doesn’t take into account already installed packages is what forces him to specify the variants every time:
It sounds to me like this is a completely different problem. Even when he specifies the variants manually, he is getting two different specs, one with python and one without. Because of this, he is unable to install visit.
@mathstuf You may want to take a look at this. It looks like a deptype problem to me. From my understanding, the problem is that when you build qt, the libxcb dependency has a build dependency on Python. But when you build VisIt, it has a normal dependency on Python. That’s why the hashes are the same but the spec is different.