spack: Error: No module named 'decimal' on environment install

Steps to reproduce

No idea why I’m experiencing this and no one else is, but any time I install an environment, I see an error message.

Error message

$ spack -d -e . install
...
==> [2022-07-08-10:54:53.744735] Regenerating environment views for /Users/ajstewart/.spack
==> [2022-07-08-10:54:53.899933] View at /Users/ajstewart/.spack/.spack-env/view does not need regeneration.
==> [2022-07-08-10:54:53.900032] Reading config file /Users/ajstewart/spack/etc/spack/defaults/modules.yaml
==> [2022-07-08-10:54:53.907032] Reading config file /Users/ajstewart/spack/etc/spack/defaults/darwin/modules.yaml
==> [2022-07-08-10:54:53.911969] 	WRITE: grep@3.7%apple-clang@13.1.6~pcre arch=darwin-monterey-m1/p6le6uv [/Users/ajstewart/spack/share/spack/modules/darwin-monterey-m1/grep-3.7-apple-clang-13.1.6-p6le6uv]
Traceback (most recent call last):
  File "/Users/ajstewart/spack/bin/spack", line 98, in <module>
    sys.exit(spack.main.main())
  File "/Users/ajstewart/spack/lib/spack/spack/main.py", line 893, in main
    return _main(argv)
  File "/Users/ajstewart/spack/lib/spack/spack/main.py", line 848, in _main
    return finish_parse_and_run(parser, cmd_name, env_format_error)
  File "/Users/ajstewart/spack/lib/spack/spack/main.py", line 876, in finish_parse_and_run
    return _invoke_command(command, parser, args, unknown)
  File "/Users/ajstewart/spack/lib/spack/spack/main.py", line 533, in _invoke_command
    return_val = command(parser, args)
  File "/Users/ajstewart/spack/lib/spack/spack/cmd/install.py", line 345, in install
    env.write()
  File "/Users/ajstewart/spack/lib/spack/spack/environment/environment.py", line 1976, in write
    spack.hooks.post_env_write(self)
  File "/Users/ajstewart/spack/lib/spack/spack/hooks/__init__.py", line 75, in __call__
    hook(*args, **kwargs)
  File "/Users/ajstewart/spack/lib/spack/spack/hooks/module_file_generation.py", line 56, in post_env_write
    _for_each_enabled(spec, 'write')
  File "/Users/ajstewart/spack/lib/spack/spack/hooks/module_file_generation.py", line 32, in _for_each_enabled
    getattr(generator, method_name)()
  File "/Users/ajstewart/spack/lib/spack/spack/modules/common.py", line 859, in write
    import jinja2
  File "/Users/ajstewart/spack/lib/spack/external/jinja2/__init__.py", line 12, in <module>
    from .environment import Environment
  File "/Users/ajstewart/spack/lib/spack/external/jinja2/environment.py", line 25, in <module>
    from .defaults import BLOCK_END_STRING
  File "/Users/ajstewart/spack/lib/spack/external/jinja2/defaults.py", line 4, in <module>
    from .tests import TESTS as DEFAULT_TESTS  # noqa: F401
  File "/Users/ajstewart/spack/lib/spack/external/jinja2/tests.py", line 3, in <module>
    import decimal
ModuleNotFoundError: No module named 'decimal'

This doesn’t make any sense, as decimal is a builtin Python module and I have no issues importing it, even within Spack:

$ spack python
Spack version 0.19.0.dev0
Python 3.9.13, Darwin arm64
>>> import decimal
>>> 

Information on your system

  • Spack: 0.19.0.dev0 (a12ce9fd59c32e650b43610925e454322995a81d)
  • Python: 3.9.13
  • Platform: darwin-monterey-m1
  • Concretizer: clingo

General information

  • I have run spack debug report and reported the version of Spack/Python/Platform
  • I have searched the issues of this repo and believe this is not a duplicate
  • I have run the failing commands in debug mode and reported the output

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 15 (8 by maintainers)

Most upvoted comments

I recently got the same error and I think I have successfully determined the cause and how to fix it.

For reference, I got this in both 0.19.0 and 0.19.1.

The problem has to do with which python is used to run spack. If a python installed by spack is earlier in PATH than the system python, spack will use that python. The problem occurs if, while installing a package, spack replaces the python that it is running on with a different one. Then, when it goes to rebuild the view, it needs to import the decimal module, it no longer exists.

Essentally, spack is rug pulling itself in this situation.

The solution is to set the environmental variable SPACK_PYTHON to the path to the system python or some other python interpreter that is guaranteed to be unaffected by whatever spack does.

Is this still an issue? I haven’t encountered this in years.

I’m going to reopen this while I think of ways to prevent it – I still think it’s an issue and we shouldn’t just have env installs result in this error.

Thanks to @frejanordsiek for diagnosing this!

Hmm, how would you propose changing how spack determines which interpreter to use? The interpreter works just fine when you first run spack, the issue is during the middle of runtime.

I also see this, FWIW. I’m not able to reproduce it by calling spack env view regenerate (even after deleting the view entirely).