staged-recipes: build-locally fails with `Command '['.scripts/run_docker_build.sh']' returned non-zero exit status 1.`

Thank you @jmp75 for your remarks. I have just tried out your command in the Ubuntu WSL and the error is resolved. Thus, the issue here is resolved. What remains open and should be discussed elsewhere is:

• Can the Shell scripts be replaced with an OS-agnostic programming language so that the Python script build-locally.py can also be started from Windows?
• Can the default docker image be replaced with an image that also supports Debian so that things just work out of the box?

I can report the same issue on a Linux Debian box. I wanted to build locally to test if and how package unit tests were run by the build process, as these rely on C compilation, before submitting a pull request.

It does grate a bit to have failing processes without a helpful error message. I’ll see if I can diagnose further with the replies in this issue, and report.

Background/context info:

Linux blah-blah-blah-machine 5.10.0-14-amd64 #1 SMP Debian 5.10.113-1 (2022-04-29) x86_64 GNU/Linux
Docker version 20.10.5+dfsg1, build 55c4c88
Python 3.9.7

python build-locallly.py linux64 :

Status: Downloaded newer image for quay.io/condaforge/linux-anvil-comp7:latest
quay.io/condaforge/linux-anvil-comp7:latest
INFO: docker run -it -v /home/per202/src/staged-recipes:/home/conda/staged-recipes -e HOST_USER_ID=1000 -e AZURE=False -e CONFIG -e CI -e DEFAULT_LINUX_VERSION quay.io/condaforge/linux-anvil-comp7 bash /home/conda/staged-recipes/.scripts/build_steps.sh
Traceback (most recent call last):
  File "/home/per202/src/staged-recipes/build-locally.py", line 101, in <module>
    main()
  File "/home/per202/src/staged-recipes/build-locally.py", line 95, in main
    run_docker_build(ns)
  File "/home/per202/src/staged-recipes/build-locally.py", line 33, in run_docker_build
    subprocess.check_call([script])
  File "/home/per202/miniconda/lib/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['.scripts/run_docker_build.sh']' returned non-zero exit status 1.

I want to bump this as I am experiencing the same error using the same setup (Windows 10 + WSL Ubuntu + Docker). I ran the build-locally.py and added some print statements to the .scripts/run_docker_build.sh to get some variable values (printed under the WARNING blkio messages). It seems like CI and DEFAULT_LINUX_VERSION were never defined.

valid configs are {'linux64_cuda112', 'linux64_cuda111', 'osx64', 'linux64_cuda102', 'win64', 'linux64', 'linux64_cuda110'}
Using linux64 configuration
ns
Namespace(config='linux64', debug=True, output_id=None)
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Docker Buildx (Docker Inc., v0.8.1-docker)
  scan: Docker Scan (Docker Inc., v0.17.0)

Server:
 Containers: 48
  Running: 0
  Paused: 0
  Stopped: 48
 Images: 7
 Server Version: 20.10.13
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2a1d4dbdb2a1030dc5b01e96fb110a9d9f150ecc
 runc version: v1.0.3-0-gf46b6ba
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.4.72-microsoft-standard-WSL2
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 12.22GiB
 Name: docker-desktop
 ID: JQLB:CDLX:INH3:4ULJ:NTQZ:FXDW:Y3OJ:MAAV:LWTJ:N26M:ML3H:VIS4
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5000
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support
DOCKER_IMAGE:  quay.io/condaforge/linux-anvil-comp7
DOCKER_RUN_ARGS:  -it
HOST_USER_ID:  1000
AZURE:  False
CONFIG:  linux64
CI:
DEFAULT_LINUX_VERSION:
PROVIDER_DIR:  .scripts
Traceback (most recent call last):
  File "/home/kongg/staged-recipes/build-locally.py", line 103, in <module>
    main()
  File "/home/kongg/staged-recipes/build-locally.py", line 97, in main
    run_docker_build(ns)
  File "/home/kongg/staged-recipes/build-locally.py", line 33, in run_docker_build
    subprocess.check_call([script])
  File "/home/kongg/miniconda3/envs/docker/lib/python3.10/subprocess.py", line 369, in check_cal
l
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['.scripts/run_docker_build.sh']' returned non-zero exit
 status 1.

Then I just ran the docker run part with the following but the container immediately exited with a 139 error code without leaving a log file.

$ docker run -it --name test \
    -v /home/<user>/staged-recipes:/home/conda/staged-recipes \
    -e HOST_USER_ID=1000 \
    -e AZURE=False \
    -e CONFIG=linux64 \
    -e CI \
    -e DEFAULT_LINUX_VERSION \
    quay.io/condaforge/linux-anvil-comp7 \
    bash \
    /home/conda/staged-recipes/.scripts/build_steps.sh

Maybe this can be a Windows + WSL2 + Docker problem, but in theory these container should work regardless of OS? Maybe @isuruf @ngam @mattwthompson could provide solution? I tried to omit or add a value to the CI/DEFAULT_LINUX_VERSION variables but that didn’t work. Thank you in advance for looking taking the time to look at this!