containerd-wasm-shims: error exporting image / operating system is not supported with deployments/k3d

When running make up from folder /deployments/k3d

...
docker buildx build --platform=wasi/wasm --load -t wasmtest_spin:latest ../../images/spin
[+] Building 0.5s (18/18) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                      0.0s
 => => transferring dockerfile: 726B                                                                                                                                                                                                                      0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                         0.0s
 => => transferring context: 2B                                                                                                                                                                                                                           0.0s
 => [internal] load metadata for docker.io/library/golang:1.19.5-bullseye                                                                                                                                                                                 0.4s
 => [internal] load metadata for docker.io/library/rust:1.67                                                                                                                                                                                              0.4s
 => [build-go 1/5] FROM docker.io/library/golang:1.19.5-bullseye@sha256:78b1726316ebfdec022fe7628f07547cafcd2d8a3934288187b87173f75925bc                                                                                                                  0.0s
 => [build 1/4] FROM docker.io/library/rust:1.67@sha256:aed98ad355d0e8303d057cc4894a7d331010980eb63c7dc46176de82670881dd                                                                                                                                  0.0s
 => [internal] load build context                                                                                                                                                                                                                         0.0s
 => => transferring context: 307B                                                                                                                                                                                                                         0.0s
 => CACHED [build 2/4] WORKDIR /opt/build                                                                                                                                                                                                                 0.0s
 => CACHED [build 3/4] COPY . .                                                                                                                                                                                                                           0.0s
 => CACHED [build 4/4] RUN rustup target add wasm32-wasi && cargo build --target wasm32-wasi --release                                                                                                                                                    0.0s
 => CACHED [stage-2 1/3] COPY --from=build /opt/build/target/wasm32-wasi/release/spin_rust_hello.wasm .                                                                                                                                                   0.0s
 => CACHED [stage-2 2/3] COPY --from=build /opt/build/spin.toml .                                                                                                                                                                                         0.0s
 => CACHED [build-go 2/5] WORKDIR /opt/build                                                                                                                                                                                                              0.0s
 => CACHED [build-go 3/5] COPY . .                                                                                                                                                                                                                        0.0s
 => CACHED [build-go 4/5] RUN curl -LO https://github.com/tinygo-org/tinygo/releases/download/v0.25.0/tinygo_0.25.0_amd64.deb && dpkg -i tinygo_0.25.0_amd64.deb                                                                                          0.0s
 => CACHED [build-go 5/5] RUN cd go-hello && tinygo build -wasm-abi=generic -target=wasi -gc=leaking -o spin_go_hello.wasm main.go                                                                                                                        0.0s
 => CACHED [stage-2 3/3] COPY --from=build-go /opt/build/go-hello/spin_go_hello.wasm .                                                                                                                                                                    0.0s
 => ERROR exporting to image                                                                                                                                                                                                                              0.0s
 => => exporting layers                                                                                                                                                                                                                                   0.0s
 => => writing image sha256:67311f8ddbc767ceb889f36a141e8333b1ccd0f40aa4144716967e6b2db2e07f                                                                                                                                                              0.0s
------
 > exporting to image:
------
ERROR: failed to solve: operating system is not supported
make: *** [Makefile:29: build-workload-images] Error 1

OS

$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian

Docker

$ docker version
Client: Docker Engine - Community
 Version:           23.0.3
 API version:       1.42
 Go version:        go1.19.7
 Git commit:        3e7cbfd
 Built:             Tue Apr  4 22:03:23 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          23.0.3
  API version:      1.42 (minimum version 1.12)
  Go version:       go1.19.7
  Git commit:       59118bf
  Built:            Tue Apr  4 22:03:23 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.20
  GitCommit:        2806fc1057397dbaeefbea0e4e17bddfbd388f38
 runc:
  Version:          1.1.5
  GitCommit:        v1.1.5-0-gf19387a
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

$ docker buildx version
github.com/docker/buildx v0.10.4 c513d34

About this issue

Original URL
State: closed
Created a year ago
Comments: 26 (19 by maintainers)

Most upvoted comments

Hi everyone,

The issue here is the docker daemon, it’s not related to buildx/buildkit, buildx manages to build the image but dockerd returns an error because it can’t store wasm images.

@KaiWalter from your docker version output I can see that on linux you have the 23.0.3 dockerd version, this one doesn’t have the needed code for you to run wasm workloads.

We don’t have a stable release with all of the containerd image store code needed for Wasm yet but you could try it out with the 24.0.0-beta.1 that has the containerd image store feature. You can download it from https://download.docker.com/linux/static/test/x86_64/docker-24.0.0-beta.1.tgz. Once downloaded you will need to edit the /etc/docker/daemon.json file (create it if it doesn’t exist) and make sure the file contains the “features” property:

{ 
  "features": { 
    "containerd-snapshotter": true 
  }
}

And then you need to:

Run the daemon with sudo ./dockerd -D -H unix:///tmp/docker.sock --data-root /tmp/dockerd --pidfile /tmp/docker.pid
Create a context that will connect to this daemon: docker context create test --docker "host=unix:///tmp/test.sock"
Use that context: docker context use test
Run a wasm workload with docker or run make up Once you are done testing Wasm you can go back to your original context: docker context use default

This will all be easier on linux once 24.0 is released (soon).

On Windows/Linux/Mac this all should work once you check the User containerd for pulling and storing feature in Docker Desktop.

rumpl on Apr 13, 2023

installed beta-2 with

curl -fsSL https://test.docker.com -o test-docker.sh
sudo sh test-docker.sh

/etc/docker/daemon.json:

{ “features”: { “containerd-snapshotter”: true } }

docker info

...
 Server Version: 24.0.0-beta.2
 Storage Driver: overlayfs
  driver-type: io.containerd.snapshotter.v1

created a wasi/wasm builder for buildx:

docker buildx create --name wasm-builder --platform wasi/wasm,linux/amd64

changed deployments/k3d/Makefile - had to fall back to image file save + load as image loading did not work

...
TEST_IMG_NAME_SPIN ?= wasmtest_spin:latest
TEST_IMG_NAME_SPIN_FILE ?= ~/wasmtest_spin.tar
TEST_IMG_NAME_SLIGHT ?= wasmtest_slight:latest
TEST_IMG_NAME_SLIGHT_FILE ?= ~/wasmtest_slight.tar
...
build-workload-images:
	docker buildx use wasm-builder
	docker buildx build --platform=wasi/wasm --load -t $(TEST_IMG_NAME_SPIN) ../../images/spin
	docker buildx build --platform=wasi/wasm --load -t $(TEST_IMG_NAME_SLIGHT) ../../images/slight
	docker buildx use default

load-workload-images: build-workload-images
	docker save $(TEST_IMG_NAME_SPIN) -o $(TEST_IMG_NAME_SPIN_FILE)
	k3d image load $(TEST_IMG_NAME_SPIN_FILE)
	docker save $(TEST_IMG_NAME_SLIGHT) -o $(TEST_IMG_NAME_SLIGHT_FILE)
	k3d image load $(TEST_IMG_NAME_SLIGHT_FILE)
...

did the trick

$ make up
...wait a while...
$ make test
curl localhost:8081/spin/hello
Hello world from Spin!curl localhost:8081/slight/hello
hello world!

meaning, that I am now unblocked for the moment and can try my own workloads with that configuration

KaiWalter on Apr 19, 2023

Cheers @squillace and @rumpl Legends!

nigelpoulton on Apr 12, 2023

Close this issue as #93 was merged

Mossaka on May 4, 2023

Run the daemon with sudo ./dockerd -D -H unix:///tmp/docker.sock --data-root /tmp/dockerd --pidfile /tmp/docker.pid Create a context that will connect to this daemon: docker context create test --docker “host=unix:///tmp/test.sock”

Should the docker context be created to “host=unix:///tmp/docker.sock” instead of “unix:///tmp/test.sock”?

Mossaka on Apr 28, 2023

I failed as well; I think there’s a beta2 out, but we are NOT quite there with moby

From: Kai Walter @.> Sent: Saturday, April 15, 2023 7:44 PM To: deislabs/containerd-wasm-shims @.> Cc: Ralph Squillace (MODULES) @.>; Mention @.> Subject: Re: [deislabs/containerd-wasm-shims] error exporting image / operating system is not supported with deployments/k3d (Issue #87)

@rumpl https://github.com/rumpl I followed your instructions above for Linux (Debian 11), needing to run dockerd and make up as sudo, but still get this message:

=> ERROR exporting to image 1.4s => => exporting layers 0.0s => => exporting manifest sha256:6de5f0ecd3e0ee17cee51cc25bb696eead01959ac29eaef6fb8f13c05efb6aca 0.0s => => exporting config sha256:508159fa344c5f41864a21b47d6646161cdddfd8ae49feb20ad74a330596d624 0.0s => => exporting attestation manifest sha256:50912c607e292b9f092e2162b3213371ebafc78b923877a7f32dbab23910c306 0.5s => => exporting manifest list sha256:7685195d70a615fa350cfaf6b26d21e8ab173480bfdc13260c66200ba02a0126 0.3s => => naming to docker.io/library/wasmtest_spin:latest 0.1s => => unpacking to docker.io/library/wasmtest_spin:latest 0.0s

exporting to image:

ERROR: failed to solve: no match for platform in manifest sha256:7685195d70a615fa350cfaf6b26d21e8ab173480bfdc13260c66200ba02a0126: not found make: *** [Makefile:29: build-workload-images] Error 1

Anything else I miss?

— Reply to this email directly, view it on GitHubhttps://github.com/deislabs/containerd-wasm-shims/issues/87#issuecomment-1509906707, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAK3K7SVFDRUYEO5DHGPXE3XBLNBRANCNFSM6AAAAAAW2SGDFM. You are receiving this because you were mentioned.Message ID: @.***>

squillace on Apr 15, 2023

I’m glad that it works for you using the tools Docker has put together.
We’re going to have to understand what needs to be done to enable this in moby buildx as well. That’s clearly the issue. I’ve just upgraded buildx and have the same thing. I’ll look into the containerd settings as well.

squillace on Apr 12, 2023

With my Windows / WSL2 combination it worked

~/src/containerd-wasm-shims/deployments/k3d$ make test
curl localhost:8081/spin/hello
Hello world from Spin!curl localhost:8081/slight/hello

I think it has something to do with this switch

KaiWalter on Apr 12, 2023

Ubuntu: github.com/docker/buildx v0.10.4 c513d34

Windows (socketed to WSL2): github.com/docker/buildx v0.10.4 c513d34049e499c53468deac6c4267ee72948f02

Windows inside WSL2 / Ubuntu: github.com/docker/buildx v0.10.4 c513d34049e499c53468deac6c4267ee72948f02

KaiWalter on Apr 12, 2023

I’m on a flight most of today, but I’ll try to take a look at the airport / in flight.

devigned on Apr 12, 2023

@Mossaka and @danbugs or @devigned if they’re around this week. I’m trying to repro now. It is possible that there’s a change in the repo just after the article release. Let’s find out!

squillace on Apr 12, 2023