west: Extension commands silently missing when `git` isn't installed

@marc-hb

1. west init -m https://github.com/nrfconnect/sdk-nrf --mr v2.1-branch

2. west update

3. Notice that you can only use the build scripts if the .git directory exists

   user: matthew in ~/tmp/zephyr
   ❯ west --help | grep build
     build:                compile a Zephyr application
   
   user: matthew in ~/tmp/zephyr
   ❯ rm -rf zephyr/.git
   
   user: matthew in ~/tmp/zephyr
   ❯ west --help | grep build
   
   

This code in interestingly not part of west.

It’s still something west interacts with, that I was misplacing blame on. I later found that West itself is deeply intertwined with .git and does similar redundant checks, in addition to the file I linked also being ran when it is sourced.

You lost me even more. Please explain much more slowly what you’re trying to achieve and include an example.

I’m trying to build Zephyr with Nix, a reproducible build system. In order to fetch the source code with west, you need to create what is called a “fixed-output-derivation”. This basically means that you fetch the content once, and then record the recursive sha256 of the directory that is created as a result of running the west commands. This recursive hash of the resulting directory is called a “NAR” (nix archive) hash. This NAR hash cannot be the same twice if we are fetching the .git folder, as git does not deterministically pack/unpack. so we have to remove it in order to fetch and store it with Nix.

This fixed-output-derivation can be thought of as a layer in Docker where you might fetch the source code. You would want that layer to be cryptographically reproducible if you were to run it again, which it cannot be if the .git remains in the result. Fixed output derivations in Nix cannot compile source code, they can only fetch data, this is an important distinction, as it prevents all sorts of cheats that would make a build process unreproducible, which is why I cannot fetch and compile in the same step. It is better for caching to fetch the source code in a separate “layer” anyway. But for as long as the .git directory is required, we cannot store it on a reproducible filesystem like the /nix/store with Nix, since the content will never be the same twice due to the aforementioned non-determinism in .git