spack: macOS M1: Spack fails to bootstrap without Xcode

I started from a clean repository updated to the latest develop branch (c2e1a12cdf44edd7f29e465bbdfdc511b97d9669).

$ spack compilers
==> Available compilers
-- apple-clang bigsur-arm64 -------------------------------------
apple-clang@12.0.0
~ $ spack spec qemu
Input spec
--------------------------------
qemu

Concretized
--------------------------------
==> Bootstrapping clingo from pre-built binaries
==> Bootstrapping clingo from sources
==> Installing gnuconfig-2021-08-14-gedovjnheiii5r4na5e3pjlekcergep6
==> No binary for gnuconfig-2021-08-14-gedovjnheiii5r4na5e3pjlekcergep6 found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/69/69b6d2868e70167ba1bdb9030b49beeb20f00b37e30825e83fd04291d96bc5f7.tar.gz
==> No patches needed for gnuconfig
==> gnuconfig: Executing phase: 'install'
==> gnuconfig: Successfully installed gnuconfig-2021-08-14-gedovjnheiii5r4na5e3pjlekcergep6
  Fetch: 0.85s.  Build: 0.19s.  Total: 1.03s.
[+] /Users/ondrej/.spack/bootstrap/store/darwin-bigsur-aarch64/apple-clang-12.0.5/gnuconfig-2021-08-14-gedovjnheiii5r4na5e3pjlekcergep6
==> Installing zlib-1.2.11-a3p23rnymc7d6ens3mk7wj75r5sucdvr
==> No binary for zlib-1.2.11-a3p23rnymc7d6ens3mk7wj75r5sucdvr found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/c3/c3e5e9fdd5004dcb542feda5ee4f0ff0744628baf8ed2dd5d66f8ca1197cb1a1.tar.gz
==> No patches needed for zlib
==> zlib: Executing phase: 'install'
==> zlib: Successfully installed zlib-1.2.11-a3p23rnymc7d6ens3mk7wj75r5sucdvr
  Fetch: 1.06s.  Build: 1.68s.  Total: 2.74s.
[+] /Users/ondrej/.spack/bootstrap/store/darwin-bigsur-aarch64/apple-clang-12.0.5/zlib-1.2.11-a3p23rnymc7d6ens3mk7wj75r5sucdvr
[+] /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8 (external python-3.8-yumcagn3zdktmhjepxgxvbokqdl24zaz)
==> Installing libiconv-1.16-fyaei4lfz3n7nzpna2vetg4a5uq7pdbk
==> No binary for libiconv-1.16-fyaei4lfz3n7nzpna2vetg4a5uq7pdbk found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/e6/e6a1b1b589654277ee790cce3734f07876ac4ccfaecbee8afa0b649cf529cc04.tar.gz
==> No patches needed for libiconv
==> libiconv: Executing phase: 'autoreconf'
==> libiconv: Executing phase: 'configure'
==> libiconv: Executing phase: 'build'
==> libiconv: Executing phase: 'install'
==> libiconv: Successfully installed libiconv-1.16-fyaei4lfz3n7nzpna2vetg4a5uq7pdbk
  Fetch: 2.18s.  Build: 18.15s.  Total: 20.34s.
[+] /Users/ondrej/.spack/bootstrap/store/darwin-bigsur-aarch64/apple-clang-12.0.5/libiconv-1.16-fyaei4lfz3n7nzpna2vetg4a5uq7pdbk
==> Installing libsigsegv-2.13-jx46dmox6wtqotnsvvcrqwx6jfuj72pc
==> No binary for libsigsegv-2.13-jx46dmox6wtqotnsvvcrqwx6jfuj72pc found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/be/be78ee4176b05f7c75ff03298d84874db90f4b6c9d5503f0da1226b3a3c48119.tar.gz
==> No patches needed for libsigsegv
==> libsigsegv: Executing phase: 'autoreconf'
==> libsigsegv: Executing phase: 'configure'
==> libsigsegv: Executing phase: 'build'
==> libsigsegv: Executing phase: 'install'
==> libsigsegv: Successfully installed libsigsegv-2.13-jx46dmox6wtqotnsvvcrqwx6jfuj72pc
  Fetch: 0.96s.  Build: 4.42s.  Total: 5.38s.
[+] /Users/ondrej/.spack/bootstrap/store/darwin-bigsur-aarch64/apple-clang-12.0.5/libsigsegv-2.13-jx46dmox6wtqotnsvvcrqwx6jfuj72pc
==> Installing berkeley-db-18.1.40-7q4rqdl4vtulxj4jx56l2mac2d2embea
==> No binary for berkeley-db-18.1.40-7q4rqdl4vtulxj4jx56l2mac2d2embea found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/0c/0cecb2ef0c67b166de93732769abdeba0555086d51de1090df325e18ee8da9c8.tar.gz
==> Applied patch /Users/ondrej/repos/spack/var/spack/repos/builtin/packages/berkeley-db/drop-docs.patch
==> Ran patch() for berkeley-db
==> berkeley-db: Executing phase: 'autoreconf'
==> berkeley-db: Executing phase: 'configure'
==> berkeley-db: Executing phase: 'build'
==> berkeley-db: Executing phase: 'install'
==> berkeley-db: Successfully installed berkeley-db-18.1.40-7q4rqdl4vtulxj4jx56l2mac2d2embea
  Fetch: 9.03s.  Build: 26.26s.  Total: 35.30s.
[+] /Users/ondrej/.spack/bootstrap/store/darwin-bigsur-aarch64/apple-clang-12.0.5/berkeley-db-18.1.40-7q4rqdl4vtulxj4jx56l2mac2d2embea
==> Installing re2c-2.2-ahy7ntwn3wjqkphsppo2cixexk62gmau
==> No binary for re2c-2.2-ahy7ntwn3wjqkphsppo2cixexk62gmau found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/0f/0fc45e4130a8a555d68e230d1795de0216dfe99096b61b28e67c86dfd7d86bda.tar.xz
==> No patches needed for re2c
==> re2c: Executing phase: 'autoreconf'
==> re2c: Executing phase: 'configure'
==> re2c: Executing phase: 'build'
==> re2c: Executing phase: 'install'
==> re2c: Successfully installed re2c-2.2-ahy7ntwn3wjqkphsppo2cixexk62gmau
  Fetch: 1.38s.  Build: 20.24s.  Total: 21.62s.
[+] /Users/ondrej/.spack/bootstrap/store/darwin-bigsur-aarch64/apple-clang-12.0.5/re2c-2.2-ahy7ntwn3wjqkphsppo2cixexk62gmau
==> Installing pkgconf-1.8.0-crd2i745cj5p6am2m5sajcycuvr3jonc
==> No binary for pkgconf-1.8.0-crd2i745cj5p6am2m5sajcycuvr3jonc found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/ef/ef9c7e61822b7cb8356e6e9e1dca58d9556f3200d78acab35e4347e9d4c2bbaf.tar.xz
==> No patches needed for pkgconf
==> pkgconf: Executing phase: 'autoreconf'
==> pkgconf: Executing phase: 'configure'
==> pkgconf: Executing phase: 'build'
==> pkgconf: Executing phase: 'install'
==> pkgconf: Successfully installed pkgconf-1.8.0-crd2i745cj5p6am2m5sajcycuvr3jonc
  Fetch: 1.07s.  Build: 5.01s.  Total: 6.08s.
[+] /Users/ondrej/.spack/bootstrap/store/darwin-bigsur-aarch64/apple-clang-12.0.5/pkgconf-1.8.0-crd2i745cj5p6am2m5sajcycuvr3jonc
==> Installing diffutils-3.8-tps25qichnyhdskh2yoimpoeersmou27
==> No binary for diffutils-3.8-tps25qichnyhdskh2yoimpoeersmou27 found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/a6/a6bdd7d1b31266d11c4f4de6c1b748d4607ab0231af5188fc2533d0ae2438fec.tar.xz
==> No patches needed for diffutils
==> diffutils: Executing phase: 'autoreconf'
==> diffutils: Executing phase: 'configure'
==> diffutils: Executing phase: 'build'
==> diffutils: Executing phase: 'install'
==> diffutils: Successfully installed diffutils-3.8-tps25qichnyhdskh2yoimpoeersmou27
  Fetch: 1.41s.  Build: 32.46s.  Total: 33.87s.
[+] /Users/ondrej/.spack/bootstrap/store/darwin-bigsur-aarch64/apple-clang-12.0.5/diffutils-3.8-tps25qichnyhdskh2yoimpoeersmou27
==> Installing m4-1.4.19-ghuewuay4dqfxf35fzbqknakd5juhl7a
==> No binary for m4-1.4.19-ghuewuay4dqfxf35fzbqknakd5juhl7a found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/3b/3be4a26d825ffdfda52a56fc43246456989a3630093cced3fbddf4771ee58a70.tar.gz
==> Fetching https://mirror.spack.io/_source-cache/archive/bf/bfdffa7c2eb01021d5849b36972c069693654ad826c1a20b53534009a4ec7a89
==> Fetching https://mirror.spack.io/_source-cache/archive/9d/9dc5fbd0d5cb1037ab1e6d0ecc74a30df218d0a94bdd5a02759a97f62daca573
==> Applied patch http://git.savannah.gnu.org/cgit/m4.git/patch/?id=a1354086
==> Applied patch http://git.savannah.gnu.org/cgit/m4.git/patch/?id=cd7f4d15
==> Ran patch() for m4
==> m4: Executing phase: 'autoreconf'
==> m4: Executing phase: 'configure'
==> m4: Executing phase: 'build'
==> m4: Executing phase: 'install'
==> m4: Successfully installed m4-1.4.19-ghuewuay4dqfxf35fzbqknakd5juhl7a
  Fetch: 1.88s.  Build: 47.45s.  Total: 49.33s.
[+] /Users/ondrej/.spack/bootstrap/store/darwin-bigsur-aarch64/apple-clang-12.0.5/m4-1.4.19-ghuewuay4dqfxf35fzbqknakd5juhl7a
==> Installing ncurses-6.2-exubvurzson2t7xztjvlg7gayp65kg7p
==> No binary for ncurses-6.2-exubvurzson2t7xztjvlg7gayp65kg7p found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/30/30306e0c76e0f9f1f0de987cf1c82a5c21e1ce6568b9227f7da5b71cbea86c9d.tar.gz
==> No patches needed for ncurses
==> ncurses: Executing phase: 'autoreconf'
==> ncurses: Executing phase: 'configure'
==> ncurses: Executing phase: 'build'
==> ncurses: Executing phase: 'install'
==> ncurses: Successfully installed ncurses-6.2-exubvurzson2t7xztjvlg7gayp65kg7p
  Fetch: 2.19s.  Build: 55.02s.  Total: 57.21s.
[+] /Users/ondrej/.spack/bootstrap/store/darwin-bigsur-aarch64/apple-clang-12.0.5/ncurses-6.2-exubvurzson2t7xztjvlg7gayp65kg7p
==> Installing bzip2-1.0.8-xoi7soo72qnpkswxbdby3a4kmip2e3ro
==> No binary for bzip2-1.0.8-xoi7soo72qnpkswxbdby3a4kmip2e3ro found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/ab/ab5a03176ee106d3f0fa90e381da478ddae405918153cca248e682cd0c4a2269.tar.gz
==> Ran patch() for bzip2
==> bzip2: Executing phase: 'install'
==> bzip2: Successfully installed bzip2-1.0.8-xoi7soo72qnpkswxbdby3a4kmip2e3ro
  Fetch: 1.22s.  Build: 1.03s.  Total: 2.25s.
[+] /Users/ondrej/.spack/bootstrap/store/darwin-bigsur-aarch64/apple-clang-12.0.5/bzip2-1.0.8-xoi7soo72qnpkswxbdby3a4kmip2e3ro
==> Installing readline-8.1-odmkrknw6jd6bu2rdomd5ivloae6gwig
==> No binary for readline-8.1-odmkrknw6jd6bu2rdomd5ivloae6gwig found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/f8/f8ceb4ee131e3232226a17f51b164afc46cd0b9e6cef344be87c65962cb82b02.tar.gz
==> Ran patch() for readline
==> readline: Executing phase: 'autoreconf'
==> readline: Executing phase: 'configure'
==> readline: Executing phase: 'build'
==> readline: Executing phase: 'install'
==> readline: Successfully installed readline-8.1-odmkrknw6jd6bu2rdomd5ivloae6gwig
  Fetch: 1.89s.  Build: 8.31s.  Total: 10.20s.
[+] /Users/ondrej/.spack/bootstrap/store/darwin-bigsur-aarch64/apple-clang-12.0.5/readline-8.1-odmkrknw6jd6bu2rdomd5ivloae6gwig
==> Installing gdbm-1.19-tcvwmvv2iq7slyhayv5wswqpt7oy3ynw
==> No binary for gdbm-1.19-tcvwmvv2iq7slyhayv5wswqpt7oy3ynw found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/37/37ed12214122b972e18a0d94995039e57748191939ef74115b1d41d8811364bc.tar.gz
==> No patches needed for gdbm
==> gdbm: Executing phase: 'autoreconf'
==> gdbm: Executing phase: 'configure'
==> gdbm: Executing phase: 'build'
==> gdbm: Executing phase: 'install'
==> gdbm: Successfully installed gdbm-1.19-tcvwmvv2iq7slyhayv5wswqpt7oy3ynw
  Fetch: 0.95s.  Build: 8.24s.  Total: 9.19s.
[+] /Users/ondrej/.spack/bootstrap/store/darwin-bigsur-aarch64/apple-clang-12.0.5/gdbm-1.19-tcvwmvv2iq7slyhayv5wswqpt7oy3ynw
==> Installing perl-5.34.0-xzib42oxhft2bgakyhhovjy5o3lvip4w
==> No binary for perl-5.34.0-xzib42oxhft2bgakyhhovjy5o3lvip4w found: installing from source
==> Warning: Failed to detect macOS command line tools version: No receipt for 'com.apple.pkg.CLTools_Executables' found at '/'.

==> Fetching https://mirror.spack.io/_source-cache/archive/55/551efc818b968b05216024fb0b727ef2ad4c100f8cb6b43fab615fa78ae5be9a.tar.gz
==> Fetching https://mirror.spack.io/_source-cache/archive/9d/9da50e155df72bce55cb69f51f1dbb4b62d23740fb99f6178bb27f22ebdf8a46.tar.gz
==> Moving resource stage
	source : /var/folders/wm/h8g03zb53_sgm6r8r8h9pmpw0000gp/T/ondrej/spack-stage/resource-cpanm-xzib42oxhft2bgakyhhovjy5o3lvip4w/spack-src/
	destination : /var/folders/wm/h8g03zb53_sgm6r8r8h9pmpw0000gp/T/ondrej/spack-stage/spack-stage-perl-5.34.0-xzib42oxhft2bgakyhhovjy5o3lvip4w/spack-src/cpanm/cpanm
==> Ran patch() for perl
==> perl: Executing phase: 'configure'
==> perl: Executing phase: 'build'
==> perl: Executing phase: 'install'
==> perl: Successfully installed perl-5.34.0-xzib42oxhft2bgakyhhovjy5o3lvip4w
  Fetch: 6.44s.  Build: 1m 30.54s.  Total: 1m 36.98s.
[+] /Users/ondrej/.spack/bootstrap/store/darwin-bigsur-aarch64/apple-clang-12.0.5/perl-5.34.0-xzib42oxhft2bgakyhhovjy5o3lvip4w
==> Installing bison-3.8.2-comfky2x72qyu32uakmlltap4sc5jfit
==> No binary for bison-3.8.2-comfky2x72qyu32uakmlltap4sc5jfit found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/06/06c9e13bdf7eb24d4ceb6b59205a4f67c2c7e7213119644430fe82fbd14a0abb.tar.gz
==> No patches needed for bison
==> bison: Executing phase: 'autoreconf'
==> bison: Executing phase: 'configure'
==> bison: Executing phase: 'build'
==> bison: Executing phase: 'install'
==> bison: Successfully installed bison-3.8.2-comfky2x72qyu32uakmlltap4sc5jfit
  Fetch: 2.27s.  Build: 34.04s.  Total: 36.31s.
[+] /Users/ondrej/.spack/bootstrap/store/darwin-bigsur-aarch64/apple-clang-12.0.5/bison-3.8.2-comfky2x72qyu32uakmlltap4sc5jfit
==> Installing openssl-1.1.1l-xq4viuhggqsoqii6osjf5k25ugjizejv
==> No binary for openssl-1.1.1l-xq4viuhggqsoqii6osjf5k25ugjizejv found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/0b/0b7a3e5e59c34827fe0c3a74b7ec8baef302b98fa80088d7f9153aa16fa76bd1.tar.gz
==> Ran patch() for openssl
==> openssl: Executing phase: 'install'
==> openssl: Successfully installed openssl-1.1.1l-xq4viuhggqsoqii6osjf5k25ugjizejv
  Fetch: 3.51s.  Build: 24.16s.  Total: 27.67s.
[+] /Users/ondrej/.spack/bootstrap/store/darwin-bigsur-aarch64/apple-clang-12.0.5/openssl-1.1.1l-xq4viuhggqsoqii6osjf5k25ugjizejv
==> Installing cmake-3.22.1-i4ydo5hgu5g5dzybpmqznbihusnz7hwq
==> No binary for cmake-3.22.1-i4ydo5hgu5g5dzybpmqznbihusnz7hwq found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/0e/0e998229549d7b3f368703d20e248e7ee1f853910d42704aa87918c213ea82c0.tar.gz
==> No patches needed for cmake
==> cmake: Executing phase: 'bootstrap'
==> cmake: Executing phase: 'build'
==> cmake: Executing phase: 'install'
==> cmake: Successfully installed cmake-3.22.1-i4ydo5hgu5g5dzybpmqznbihusnz7hwq
  Fetch: 3.90s.  Build: 3m 22.81s.  Total: 3m 26.71s.
[+] /Users/ondrej/.spack/bootstrap/store/darwin-bigsur-aarch64/apple-clang-12.0.5/cmake-3.22.1-i4ydo5hgu5g5dzybpmqznbihusnz7hwq
==> Installing clingo-bootstrap-spack-7ctouvxrqj6vs2mw2txgl5jfsqfrnpuc
==> No binary for clingo-bootstrap-spack-7ctouvxrqj6vs2mw2txgl5jfsqfrnpuc found: installing from source
==> Error: RuntimeError: Unable to locate Python3.framework/Versions/3.8/Python3 libraries in /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib

/Users/ondrej/repos/spack/var/spack/repos/builtin/packages/python/package.py:828, in libs:
        825                return LibraryList(os.path.join(macos_developerdir, ldlibrary))
        826            else:
        827                msg = 'Unable to locate {0} libraries in {1}'
  >>    828                raise RuntimeError(msg.format(ldlibrary, libdir))
        829        else:
        830            library = self.config_vars['LIBRARY']
        831


==> Error: cannot bootstrap the "clingo" Python module from spec "clingo-bootstrap@spack+python %apple-clang target=aarch64" due to the following failures:
    'spack-install' raised InstallError: Terminating after first install failure: RuntimeError: Unable to locate Python3.framework/Versions/3.8/Python3 libraries in /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib
    Please run `spack -d spec zlib` for more verbose error messages

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 22 (22 by maintainers)

Most upvoted comments

My understanding is that the only dependency on Xcode is via depending on Python from it. But it seems that Spack works with any Python package, such as the one from Conda. There is Python installed using commandline utilities, but the paths must be adjusted in Spack for that. Finally, Spack insists on creating some directories in the Python installation, which I don’t quite understand why, and that fails too obviously. But those are all technical issues that are not related to Xcode in my opinion.

I see. Thanks for the feedback. The reason for me to use Conda and Spack on macOS is to bypass the default way of installing packages, which I do not find user friendly or maintainable. Installing Xcode is one of those big packages that I do not like installing. I just like to take the stock macOS as it comes, and just do minimal changes to it. The command line utilities are needed (it seems), so I am fine with that.

I tried to look what it takes to install Xcode, and it is asking me to sign with my Apple ID. I don’t want to do that either.

The good news is that Conda works, and apparently it is able to create the initial environment for Spack to operate in, so that’s fine and I’ll just use Conda instead of Xcode. I do think Spack itself should not require Xcode — I got it working with an earlier version of Spack, the latest version seems even more depending on Xcode, but I don’t think there is any technical reason (necessity) for that.

I mean: installing the 12.7 GB Xcode just to have the python framework in the expected location is a hefty price to pay.

Not sure if/how we can lift the requirement, but as far as I remember we always had this documented as:

on macOS, XCode is required.

at this page

I don’t think my conda installation is interfering with Spack. It doesn’t alter any paths unless I do “conda activate something”.

The apple developer tools documentation:

This package enables UNIX-style development via Terminal by installing command line developer tools, as well as macOS SDK frameworks and headers. Many useful tools are included, such as the Apple LLVM compiler, linker, and Make. If you use Xcode, these tools are also embedded within the Xcode IDE.

So I guess the distinction may be that the CLT are for command-line-focused (no GUI components) development and Xcode proper is for development that could involve the gui or other Apple platforms.

Interesting, well the next time I set up a computer I’ll try not installing xcode and see what errors crop up. I think the assumption is generally that if you’re “developing” (i.e. building code) on mac then Xcode is needed — homebrew and conda can likely get away with this because they distribute prebuilt binaries. Perhaps Spack can get to that point as well.

Yes, I am developing on macOS. The CommandLineTools tools have the compiler (clang). I haven’t done GUI, I just develop C++ and Fortran applications, and they all work with just CommandLineTools. I guess I like to keep my system minimal. So far Spack is the first project that requires it.

@certik I found it much easier to bootstrap with homebrew: brew install gcc, spack compiler find gcc

On an M1 Max, I still get exactly the same error as above, but unfortunately the patch above does not fix it, I get further errors. The following patch gets it to start compiling clingo-bootstrap-spack-fwqxswj4iz5tyic2sjguthrzmundqq3z:

diff --git a/var/spack/repos/builtin/packages/python/package.py b/var/spack/repos/builtin/packages/python/package.py
index 222b3f97a9..f86748f2a3 100644
--- a/var/spack/repos/builtin/packages/python/package.py
+++ b/var/spack/repos/builtin/packages/python/package.py
@@ -868,6 +868,7 @@ def libs(self):
         # The system Python installation on macOS and Homebrew installations
         # install libraries into a Frameworks directory
         frameworkprefix = self.config_vars['PYTHONFRAMEWORKPREFIX']
+        frameworkprefix = "/Library/Developer/CommandLineTools/Library/Frameworks/"
 
         # Get the active Xcode environment's Framework location.
         macos_developerdir = os.environ.get('DEVELOPER_DIR')
@@ -906,6 +907,7 @@ def libs(self):
     @property
     def headers(self):
         config_h = self.config_vars['config_h_filename']
+        config_h = "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Headers/pyconfig.h"
 
         if os.path.exists(config_h):
             headers = HeaderList(config_h)
@@ -1100,9 +1102,9 @@ def setup_dependent_package(self, module, dependent_spec):
         self.spec.home = self.home
 
         # Make the site packages directory for extensions
-        if dependent_spec.package.is_extension:
-            mkdirp(module.python_platlib)
-            mkdirp(module.python_purelib)
+        #if dependent_spec.package.is_extension:
+            #mkdirp(module.python_platlib)
+            #mkdirp(module.python_purelib)
 
     # ========================================================================
     # Handle specifics of activating and deactivating python modules.

But it fails with:

==> Installing clingo-bootstrap-spack-fwqxswj4iz5tyic2sjguthrzmundqq3z
==> No binary for clingo-bootstrap-spack-fwqxswj4iz5tyic2sjguthrzmundqq3z found: installing from source
==> Ran patch() for clingo-bootstrap
==> clingo-bootstrap: Executing phase: 'cmake'
==> Error: ProcessError: Command exited with status 1:
    'cmake' '-G' 'Unix Makefiles' '-DCMAKE_INSTALL_PREFIX:STRING=/Users/ondrej/.spack/bootstrap/store/darwin-monterey-aarch64/apple-clang-13.0.0/clingo-bootstrap-spack-fwqxswj4iz5tyic2sjguthrzmundqq3z' '-DCMAKE_BUILD_TYPE:STRING=Release' '-DCMAKE_INTERPROCEDURAL_OPTIMIZATION:BOOL=OFF' '-DCMAKE_VERBOSE_MAKEFILE:BOOL=ON' '-DCMAKE_FIND_FRAMEWORK:STRING=LAST' '-DCMAKE_FIND_APPBUNDLE:STRING=LAST' '-DCMAKE_INSTALL_RPATH_USE_LINK_PATH:BOOL=OFF' '-DCMAKE_INSTALL_RPATH:STRING=/Users/ondrej/.spack/bootstrap/store/darwin-monterey-aarch64/apple-clang-13.0.0/clingo-bootstrap-spack-fwqxswj4iz5tyic2sjguthrzmundqq3z/lib;/Users/ondrej/.spack/bootstrap/store/darwin-monterey-aarch64/apple-clang-13.0.0/clingo-bootstrap-spack-fwqxswj4iz5tyic2sjguthrzmundqq3z/lib64;/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib' '-DCMAKE_PREFIX_PATH:STRING=/Users/ondrej/.spack/bootstrap/store/darwin-monterey-aarch64/apple-clang-13.0.0/re2c-2.2-l2xalq74s55fxw3f7mui2t7kawp7tkcp;/Users/ondrej/.spack/bootstrap/store/darwin-monterey-aarch64/apple-clang-13.0.0/cmake-3.22.2-hvwqqlibhn6djb5suglgi2yoxkisobwo;/Users/ondrej/.spack/bootstrap/store/darwin-monterey-aarch64/apple-clang-13.0.0/bison-3.8.2-cc2tfmnwhcf6uu6mw2m2qilokrzoijne;/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8' '-DCLINGO_REQUIRE_PYTHON=ON' '-DCLINGO_BUILD_WITH_PYTHON=ON' '-DPYCLINGO_USER_INSTALL=OFF' '-DPYCLINGO_USE_INSTALL_PREFIX=ON' '-DCLINGO_BUILD_WITH_LUA=OFF' '-DCLINGO_BUILD_PY_SHARED:STRING=OFF' '-DPython_EXECUTABLE:STRING=/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/bin/python3.8' '-DPython_INCLUDE_DIR:STRING=/Library/Python/3.8/include' '-DCLINGO_BUILD_APPS:STRING=OFF' '/var/folders/8w/5mqfkhqn1nj0tw97g3hl1nyh0000gn/T/ondrej/spack-stage/spack-stage-clingo-bootstrap-spack-fwqxswj4iz5tyic2sjguthrzmundqq3z/spack-src'

1 error found in build log:
     9     -- Detecting C compile features - done
     10    -- Detecting CXX compiler ABI info
     11    -- Detecting CXX compiler ABI info - done
     12    -- Check for working CXX compiler: /Users/ondrej/repos/spack/lib/spa
           ck/env/clang/clang++ - skipped
     13    -- Detecting CXX compile features
     14    -- Detecting CXX compile features - done
  >> 15    CMake Error at /Users/ondrej/.spack/bootstrap/store/darwin-monterey-
           aarch64/apple-clang-13.0.0/cmake-3.22.2-hvwqqlibhn6djb5suglgi2yoxkis
           obwo/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:23
           0 (message):
     16      Could NOT find Python (missing: Interpreter Development Developmen
           t.Module
     17      Development.Embed) (found version "2.7")
     18    Call Stack (most recent call first):
     19      /Users/ondrej/.spack/bootstrap/store/darwin-monterey-aarch64/apple
           -clang-13.0.0/cmake-3.22.2-hvwqqlibhn6djb5suglgi2yoxkisobwo/share/cm
           ake-3.22/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAI
           LURE_MESSAGE)
     20      /Users/ondrej/.spack/bootstrap/store/darwin-monterey-aarch64/apple
           -clang-13.0.0/cmake-3.22.2-hvwqqlibhn6djb5suglgi2yoxkisobwo/share/cm
           ake-3.22/Modules/FindPython.cmake:561 (find_package_handle_standard_
           args)
     21      CMakeLists.txt:130 (find_package)

See build log for details:
  /var/folders/8w/5mqfkhqn1nj0tw97g3hl1nyh0000gn/T/ondrej/spack-stage/spack-stage-clingo-bootstrap-spack-fwqxswj4iz5tyic2sjguthrzmundqq3z/spack-build-out.txt

==> Error: cannot bootstrap the "clingo" Python module from spec "clingo-bootstrap@spack+python %apple-clang target=aarch64" due to the following failures:
    'spack-install' raised InstallError: Terminating after first install failure: ProcessError: Command exited with status 1:
    'cmake' '-G' 'Unix Makefiles' '-DCMAKE_INSTALL_PREFIX:STRING=/Users/ondrej/.spack/bootstrap/store/darwin-monterey-aarch64/apple-clang-13.0.0/clingo-bootstrap-spack-fwqxswj4iz5tyic2sjguthrzmundqq3z' '-DCMAKE_BUILD_TYPE:STRING=Release' '-DCMAKE_INTERPROCEDURAL_OPTIMIZATION:BOOL=OFF' '-DCMAKE_VERBOSE_MAKEFILE:BOOL=ON' '-DCMAKE_FIND_FRAMEWORK:STRING=LAST' '-DCMAKE_FIND_APPBUNDLE:STRING=LAST' '-DCMAKE_INSTALL_RPATH_USE_LINK_PATH:BOOL=OFF' '-DCMAKE_INSTALL_RPATH:STRING=/Users/ondrej/.spack/bootstrap/store/darwin-monterey-aarch64/apple-clang-13.0.0/clingo-bootstrap-spack-fwqxswj4iz5tyic2sjguthrzmundqq3z/lib;/Users/ondrej/.spack/bootstrap/store/darwin-monterey-aarch64/apple-clang-13.0.0/clingo-bootstrap-spack-fwqxswj4iz5tyic2sjguthrzmundqq3z/lib64;/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib' '-DCMAKE_PREFIX_PATH:STRING=/Users/ondrej/.spack/bootstrap/store/darwin-monterey-aarch64/apple-clang-13.0.0/re2c-2.2-l2xalq74s55fxw3f7mui2t7kawp7tkcp;/Users/ondrej/.spack/bootstrap/store/darwin-monterey-aarch64/apple-clang-13.0.0/cmake-3.22.2-hvwqqlibhn6djb5suglgi2yoxkisobwo;/Users/ondrej/.spack/bootstrap/store/darwin-monterey-aarch64/apple-clang-13.0.0/bison-3.8.2-cc2tfmnwhcf6uu6mw2m2qilokrzoijne;/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8' '-DCLINGO_REQUIRE_PYTHON=ON' '-DCLINGO_BUILD_WITH_PYTHON=ON' '-DPYCLINGO_USER_INSTALL=OFF' '-DPYCLINGO_USE_INSTALL_PREFIX=ON' '-DCLINGO_BUILD_WITH_LUA=OFF' '-DCLINGO_BUILD_PY_SHARED:STRING=OFF' '-DPython_EXECUTABLE:STRING=/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/bin/python3.8' '-DPython_INCLUDE_DIR:STRING=/Library/Python/3.8/include' '-DCLINGO_BUILD_APPS:STRING=OFF' '/var/folders/8w/5mqfkhqn1nj0tw97g3hl1nyh0000gn/T/ondrej/spack-stage/spack-stage-clingo-bootstrap-spack-fwqxswj4iz5tyic2sjguthrzmundqq3z/spack-src'
    Please run `spack -d spec zlib` for more verbose error messages

I don’t now how to fix that.

Spack needs to build its own Python, it should not rely on system Python like this. Also note that the lines mkdirp(module.python_platlib) were trying to create a directory in /Library/... which is unacceptable. Spack should not modify any system files, it should install everything locally without root access.

Hmm I’m guessing we need to build bootstrap binaries for darwin-arm64. The missing python framework may be a separate issue.

This error can be fixed by:

diff --git a/var/spack/repos/builtin/packages/python/package.py b/var/spack/repos/builtin/packages/python/package.py
index 141d5c703a..1cc071eefd 100644
--- a/var/spack/repos/builtin/packages/python/package.py
+++ b/var/spack/repos/builtin/packages/python/package.py
@@ -802,6 +802,7 @@ def libs(self):
         # The system Python installation on macOS and Homebrew installations
         # install libraries into a Frameworks directory
         frameworkprefix = self.config_vars['PYTHONFRAMEWORKPREFIX']
+        frameworkprefix = "/Library/Developer/CommandLineTools/Library/Frameworks/"
 
         # Get the active Xcode environment's Framework location.
         macos_developerdir = os.environ.get('DEVELOPER_DIR')

As such, I suspect that Spack requires Xcode, while I only have the command line utilities installed. Honestly, things should just work with the command line utilities, as everything else seems to work for me that way.