spack: Inconsistent handling of shortened versions
Spack seems to be inconsistently handling shorted versions.
depends_on
✔️ Edited by @haampie 2021-09-22: the issue here is no longer relevant, because depends_on('python@2.7') means the python version is in the closed-open range [2.7, 2.8).
When used in a depends_on statement, if I say:
depends_on('python@2.7')
Spack will install exactly Python 2.7, which will crash because 2.7 doesn’t exist. I would instead expect it to install the latest 2.7.X. This is a long-standing bug documented in #921 and #1035.
As of now, the workaround has been to explicitly turn this into a version range:
depends_on('python@2.7.0:2.7.999')
This works, but is unnecessarily wordy.
when=
✔️ Edited by @haampie 2021-09-22: This issue is fixed, since we can write when='@0.2:0.2.0', and the meaning is consistent with depends_on.
When used in a when= argument, if I say:
depends_on('r@3.4.0:3.4.9', when='@0.2')
this will apply to both versions 0.2 and 0.2.2, as documented in #8950. Similarly, if I say:
patch('foo.patch', when='@3.2')
this patch will be applied to both versions 3.2 and 3.2.1, as documented in #8320 and #8432.
As of now, there is no workaround for this problem. This may be fixed by #8320.
Compiler versions
This one is the worst of all. Let’s say I have a GCC 7.2.0 compiler in my compilers.yaml. There are many other GCC compilers, but this is the only GCC 7 compiler I have. If I say:
$ spack install zlib %gcc@7
Spack will correctly deduce that I want it to use my GCC 7.2.0 compiler, but it will resolve the compiler as %gcc@7 in my spec, and install to the darwin-highsierra-x86_64/gcc-7 directory. The installation succeeds, but if I rerun:
$ spack install foo %gcc@7.2.0 ^zlib
Spack needs to reinstall zlib with the new GCC 7.2.0 compiler. I believe this bug is undocumented. In fact, I think it used to work properly. I used it all the time when I had multiple GCC compilers installed and I wanted to specify one.
The workaround is to use a version range, like %gcc@7:.
Proposal
All of these situations should be handled identically. Otherwise, it’s confusing what behavior to expect. I propose the following behavior.
If a package.py/compilers.yaml contains both 3.2 and 3.2.1, 3.2 should refer to exactly 3.2 only.
If a package.py/compilers.yaml only contains 3.2.1, 3.2 should refer to any 3.2.X version.
@tgamblin @baberlevi @PDoakORNL @cyrush @davydden @goxberry @eschnett
About this issue
- Original URL
- State: closed
- Created 8 years ago
- Reactions: 4
- Comments: 26 (26 by maintainers)
There are two levels we should think about here.
One is the user facing level, at this point it really isn’t up to us to tell users how they have to think! We consistently see that
@1is expected to match everything from@1.0.0.betaup to (but not including)2.0. Even though I agree with @davydden that it is problematic that resolution of@1depends on whether aversion('1'does or does not exist, it is too natural to expect it to work and thereforespackshould support this. We may be able to train our users to do it our way ™️ (i.e. provide an error and suggest the correct version), but is this really worth it? Or even is it less work than just implementing the matching correctly?But on the other hand we have how we internally handle versions. Here we are completely free to do as we want (with sensible constructions being easier to maintain, but there is no “user”-catchall argument). Here I would say that we should always use the versions as they are declared in
package.py. I.e. all spack-generated folder structure should contain the version frompackage.pyand NOT the user provided strings.@adamjstewart how is “being able to install versions even if they don’t exist in the
package.py.” supposed to work? The one situation that I can think about right now is, that we install a package from one branch of spack and then switch to a branch that does not have the installed version of this package. But in that case the database “knows” the version, so no need to go with the user-provided string for internal useI was going to suggest creating a constraint like
2019.2:2019.99(which I thought was analogous to the suggestion in https://github.com/spack/spack/issues/13831#issuecomment-558854712) but after trying that myself I see it doesn’t work.So the problem you are discussing is more than an inconsistency: it’s a gap in the model’s ability to properly specify constraints (so I think in fact https://github.com/spack/spack/issues/8957#issuecomment-560568209 could be considered separate). I don’t think addressing your issue specifically will be very difficult although it likely won’t address the broader issue here.
@tgamblin @adamjstewart @scheibelp @becker33
Well, I thought about opening a new issue and pretending not to remember this one, just to see if that would get more attention. 😃 But clearly, Adam reported the same issue above.
And to be clear, this is not an issue of interpreting
2019insidewhen().I tried
spack install intel-tbb @2019and the concretizer picks rev exactly 2019, not 2019.8. (This much is correct.) But as Adam reported, the problem is that2019satisfies both of these:So, rev 2019 is both >= 2019.6 and <= 2019.2.
Basically, this makes it impossible to use version ranges for any package that has variable-length versions. There is no valid order where this makes sense.
IMO, this is a clear bug, seperate from how you want to interpret
when='@2019'. (One reason I always specify a full version inside when.)I was hoping this would be a small and isolated fix and could be done before working out the full details of how to interpret shortened version names.
Btw, does the full fix for this fall under the domain of the new concretizer? Is that what we’re waiting on?
@healther it becomes more annoying when you’re “installing” an external version that may be too old to download/checksum.
Ah okay, I see. Not sure, how useful this use case really is… Changing
package.pyis a rather low bar to add new versions isn’t it?I would also be fine with an error, although @tgamblin was a strong proponent of being able to install versions even if they don’t exist in the
package.py. If we want to continue to support this, I think Spack should check which versions exist in a package before determining what version is actually requested.no, of course not, what I propose is to throw an error.
EDIT: I see the merits of your approach, though.