spack: spec.satisfies needs to be refactored (behaves wrongly inside packages)

This may be the root of some weird behaviors seen around, given that currently this pattern is used in more than 40 packages. Short story: ~never use this inside packages:~ this line behaves wrongly in packages:

spec.satisfies('^foo')  # or similar checks

while this behaves correctly:

'^foo' in spec

The point is that the first call is meant to be used inside the concretizer and has a weaker semantic on enforcing the constraint. The second call has a stronger semantic, and is equivalent to calling:

for s in self.traverse():
    if s.satisfies('^foo', strict=True):
        return True

where:

  1. we recurse on the spec
  2. we activate the strict=True parameter which enforces the strong semantic

In other words (@tgamblin ™️ ) the first call means that spec could satisfy ^foo eventually. The second that spec can satisfy ^foo right now.

@scheibelp @becker33 @tgamblin Can you confirm that this could be a potential issue for packages?

About this issue

  • Original URL
  • State: closed
  • Created 10 years ago
  • Reactions: 2
  • Comments: 18 (18 by maintainers)

Commits related to this issue

Most upvoted comments

Another question kind of brought up by the last one: should Spec have an __iter__ method equivalent to spec.traverse() with no arguments? It would allow:

    any('+debug' in d for d in spec)

You would still need spec.traverse() for when you want something other than the default traversal over dependencies (different order, etc.)

I would suggest these semantics:

  1. spec.satisfies(other): same as current satisfies, but always strict. Concretizer uses could_satisfy. I actually have a branch where I started that refactor.
  2. other in spec: I like this notation a lot, and I think it makes the code more readable, both when you’re checking whether some package is in the DAG, and when you’re checking whether the spec itself satisfies an anonymous constraint. The most intuitive semantics for me would be:
    if other.name:
        return any(d.satisfies(other) for d in spec.traverse()
    else:
        return spec.satisfies(other)
    
  3. could_satisfy: new method for use by the concretizer, equivalent to satisfies with strict=False.

This makes satisfies do what you’d expect, and it makes __contains__ clear because it only applies to dependencies when you’re explicit about which one. If you want to check an anonymous spec against everything in the DAG, you have to be explicit and say:

    any('+debug' in d for d in spec.traverse())    # or
    any(d.satisfies('+debug') for d in spec.traverse())

I agree with @citibeth https://github.com/citibeth except on point 3.

I was throwing out 1,2, and 3 as brainstorm alternatives. I would not expect we implement more than one of them.

I think we should not augment the redundancy in the public API. If:

‘^foo’ in spec

is equivalent to

spec.satisfies(‘^foo’)

then one of the two must disappear in the long run.

I think that spec.satisfies() feels more clear and explicit. But using __contains__() is so convenient… If I had to vote for getting rid of one, I’d get rid of __contains__() because it’s less clear.

The main issue here is that satisfies() was written to be used in

concretization, where you want to know whether something could be satisfied,

Then maybe it’s mis-named, and the version used in concretization should be called could_satisfy().

OK, this is really subtle… It looks like a booby trap to me. I suggest we think through ways to get rid of the booby trap, instead of just documenting it. For example:

  1. Change the default for spec.satisfies from strict=False to strict=True. Then users can blithely use it in package.py without repercussions.

  2. Rename spec.satisfies to spec._satisfies, it is therefore marked for internal use only.

  3. Remove the strict parameter from spec.satisfies(), and have the behavior of the new spec.satisfies default to strict=True. Then create a new spec.satisfies_permissive(), which is equivalent to strict=False.

Finally done in #35681

@melven this issue predates the environments feature: can you open a new issue including the commands, environment, and stack trace (when running with spack -d)?

I agree with @citibeth except on point 3. I think we should not augment the redundancy in the public API. If:

'^foo' in spec

is equivalent to

spec.satisfies('^foo')

then one of the two must disappear in the long run.

If instead we want a function that peeks only at this node in the DAG we should rename satisfy to something else. I don’t think anybody used it on purpose to search only the current node in the DAG without traversing dependencies.

I agree with @citibeth that this is a bug that should be fixed rather than advice we should give to packagers. I’ll look at what it would take to change.