spack: Infinite recursion in spack spec

When I clone a fresh copy of Spack and run:

$ spack spec libxext
Input spec
--------------------------------
libxext

Normalized
--------------------------------
libxext
    ^libx11@1.6:
        ^inputproto
            ^pkg-config@0.9.0:
            ^util-macros
        ^kbproto
        ^libxcb@1.1.92:
            ^libpthread-stubs
            ^libxau@0.99.2:
                ^xproto@7.0.17:
            ^libxdmcp
            ^python@2:2.8
                ^bzip2
                ^ncurses
                ^openssl
                    ^zlib
                ^readline
                ^sqlite
            ^xcb-proto
        ^xextproto@7.1.99:
        ^xtrans

Concretized
--------------------------------
Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in <type 'exceptions.TypeError'> ignored
Exception RuntimeError: 'maximum recursion depth exceeded while calling a Python object' in <type 'exceptions.TypeError'> ignored
Traceback (most recent call last):
  File "./spack", line 198, in <module>
    main()
  File "./spack", line 175, in main
    return_val = command(parser, args)
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/cmd/spec.py", line 78, in spec
    spec.concretize()
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/spec.py", line 1507, in concretize
    changes = (self.normalize(force),
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/spec.py", line 1809, in normalize
    any_change = self._normalize_helper(visited, spec_deps, provider_index)
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/spec.py", line 1763, in _normalize_helper
    pkg_dep, deptypes, visited, spec_deps, provider_index)
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/spec.py", line 1734, in _merge_dependency
    visited, spec_deps, provider_index)
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/spec.py", line 1763, in _normalize_helper
    pkg_dep, deptypes, visited, spec_deps, provider_index)
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/spec.py", line 1734, in _merge_dependency
    visited, spec_deps, provider_index)
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/spec.py", line 1763, in _normalize_helper
    pkg_dep, deptypes, visited, spec_deps, provider_index)
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/spec.py", line 1734, in _merge_dependency
    visited, spec_deps, provider_index)
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/spec.py", line 1758, in _normalize_helper
    pkg_dep = self._evaluate_dependency_conditions(dep_name)
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/spec.py", line 1629, in _evaluate_dependency_conditions
    sat = self.satisfies(when_spec, strict=True)
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/spec.py", line 2058, in satisfies
    return self.satisfies_dependencies(other, strict=deps_strict)
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/spec.py", line 2087, in satisfies_dependencies
    self_index = ProviderIndex(self.traverse(), restrict=True)
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/provider_index.py", line 87, in __init__
    self.update(spec)
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/provider_index.py", line 99, in update
    pkg = spec.package
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/spec.py", line 945, in package
    return spack.repo.get(self)
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/repository.py", line 75, in converter
    return function(self, spec_like, *args, **kwargs)
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/repository.py", line 318, in get
    return self.repo_for_pkg(spec).get(spec)
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/repository.py", line 75, in converter
    return function(self, spec_like, *args, **kwargs)
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/repository.py", line 567, in get
    key = hash(spec)
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/llnl/util/lang.py", line 254, in <lambda>
    setter('__hash__', lambda self: hash(self._cmp_key()))
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/spec.py", line 2330, in _cmp_key
    for name, d in sorted(self._dependencies.items()))
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/spec.py", line 2330, in <genexpr>
    for name, d in sorted(self._dependencies.items()))
...
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/spec.py", line 2330, in _cmp_key
    for name, d in sorted(self._dependencies.items()))
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/spec.py", line 2330, in <genexpr>
    for name, d in sorted(self._dependencies.items()))
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/llnl/util/lang.py", line 254, in <lambda>
    setter('__hash__', lambda self: hash(self._cmp_key()))
  File "/blues/gpfs/home/ajstewart/spack/lib/spack/spack/spec.py", line 2330, in _cmp_key
    for name, d in sorted(self._dependencies.items()))
TypeError: object of type 'generator' has no len()

This also occurs for me with other very large packages like dealii. Interestingly enough, I am unable to reproduce this on another computer. I’ve tried newer versions of Python but no luck. This definitely wasn’t a problem until recently. Any ideas?

About this issue

  • Original URL
  • State: closed
  • Created 10 years ago
  • Comments: 24 (24 by maintainers)

Most upvoted comments

Closing since the feature request that would solve this issue (circular dependencies in the DAG) is tracked in #2498

@ax3l Can you reproduce that in a docker container? That will help a lot the debugging, as I am currently not able to reproduce this.

@ax3l at first glance this error appears to be different: the issue as reported in the initial description is triggered by concretization (from the spack spec command); this is triggered by the environment script, and (based on the message itself) config parsing.

If you agree, could you open a new issue, which highlights the specific part that gets repeated:

  File "/home/axel/src/spack/lib/spack/llnl/util/lang.py", line 556, in __getattr__
    return getattr(self.instance, name)
  [Previous line repeated 318 more times]

(in which case I will collapse this and https://github.com/spack/spack/issues/2495#issuecomment-419636297)

That’s because we removed a couple Python build dependencies to prevent it. See #4145.

@alalazo Is it just because python+tk depends on tk, which depends on python+tk?

Identified the difference between computers. On one computer, I have the following in my packages.yaml:

packages:
  python:
    variants: +tk

the other does not. When I copy this to the other computer, it breaks all of a sudden.