podman: Running ubi9 on Apple silicon breaks on unsupported x86_64-v2 instruction set
Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)
/kind bug
Description
I’m trying to build and run an AMD64 ubi9 image on Apple silicon. I have installed qemu-user-binfmt in the Fedora CoreOS VM, but when I start a podman build with a Containerfile that starts with FROM registry.access.redhat.com/ubi9/ubi-minimal:latest, the build fails with
Fatal glibc error: CPU does not support x86-64-v2
-
On Apple silicon, brew install podman, set up a VM
-
Try and run a build with a Containerfile that has a
FROM registry.access.redhat.com/ubi9/ubi-minimal:latestand with--arch amd64 -
See error
Describe the results you received:
✗ podman build --arch amd64 -f Containerfile -t testing:0.1 .
STEP 1/7: FROM registry.access.redhat.com/ubi9/ubi-minimal:latest
Trying to pull registry.access.redhat.com/ubi9/ubi-minimal:latest...
Getting image source signatures
Checking if image destination supports signatures
Copying blob sha256:743b2fff6512b752ffd624ae8b45a951ca8378f3d84a912061cdaf508db1a29d
Copying blob sha256:961425e673a750fb2d17875c2a4bf5a319c8681e4fdf81a5a6e2d819cef95e35
Copying config sha256:4a8128b051b8bfafcc6bd540db324a5244aff1ef7bd9f6dda7ecd10680c4fbbe
Writing manifest to image destination
Storing signatures
STEP 2/7: RUN microdnf install -y python3 python3-pip
Fatal glibc error: CPU does not support x86-64-v2
Error: error building at STEP "RUN microdnf install -y python3 python3-pip": error while running runtime: exit status 127
Describe the results you expected: Successful build
Additional information you deem important (e.g. issue happens only occasionally):
Output of podman version:
Client: Podman Engine
Version: 4.2.0
API Version: 4.2.0
Go Version: go1.18.5
Built: Wed Aug 10 22:46:05 2022
OS/Arch: darwin/arm64
Server: Podman Engine
Version: 4.2.0
API Version: 4.2.0
Go Version: go1.18.4
Built: Thu Aug 11 16:43:11 2022
OS/Arch: linux/arm64
Output of podman info:
host:
arch: arm64
buildahVersion: 1.27.0
cgroupControllers:
- cpu
- io
- memory
- pids
cgroupManager: systemd
cgroupVersion: v2
conmon:
package: conmon-2.1.0-2.fc36.aarch64
path: /usr/bin/conmon
version: 'conmon version 2.1.0, commit: '
cpuUtilization:
idlePercent: 92.23
systemPercent: 6.17
userPercent: 1.6
cpus: 1
distribution:
distribution: fedora
variant: coreos
version: "36"
eventLogger: journald
hostname: localhost.localdomain
idMappings:
gidmap:
- container_id: 0
host_id: 1000
size: 1
- container_id: 1
host_id: 100000
size: 1000000
uidmap:
- container_id: 0
host_id: 502
size: 1
- container_id: 1
host_id: 100000
size: 1000000
kernel: 5.18.18-200.fc36.aarch64
linkmode: dynamic
logDriver: journald
memFree: 1673756672
memTotal: 2051903488
networkBackend: netavark
ociRuntime:
name: crun
package: crun-1.5-1.fc36.aarch64
path: /usr/bin/crun
version: |-
crun version 1.5
commit: 54ebb8ca8bf7e6ddae2eb919f5b82d1d96863dea
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
os: linux
remoteSocket:
exists: true
path: /run/user/502/podman/podman.sock
security:
apparmorEnabled: false
capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
rootless: true
seccompEnabled: true
seccompProfilePath: /usr/share/containers/seccomp.json
selinuxEnabled: true
serviceIsRemote: true
slirp4netns:
executable: /usr/bin/slirp4netns
package: slirp4netns-1.2.0-0.2.beta.0.fc36.aarch64
version: |-
slirp4netns version 1.2.0-beta.0
commit: 477db14a24ff1a3de3a705e51ca2c4c1fe3dda64
libslirp: 4.6.1
SLIRP_CONFIG_VERSION_MAX: 3
libseccomp: 2.5.3
swapFree: 0
swapTotal: 0
uptime: 0h 2m 56.00s
plugins:
authorization: null
log:
- k8s-file
- none
- passthrough
- journald
network:
- bridge
- macvlan
volume:
- local
registries:
search:
- docker.io
store:
configFile: /var/home/core/.config/containers/storage.conf
containerStore:
number: 0
paused: 0
running: 0
stopped: 0
graphDriverName: overlay
graphOptions: {}
graphRoot: /var/home/core/.local/share/containers/storage
graphRootAllocated: 106825756672
graphRootUsed: 2977878016
graphStatus:
Backing Filesystem: xfs
Native Overlay Diff: "true"
Supports d_type: "true"
Using metacopy: "false"
imageCopyTmpDir: /var/tmp
imageStore:
number: 1
runRoot: /run/user/502/containers
volumePath: /var/home/core/.local/share/containers/storage/volumes
version:
APIVersion: 4.2.0
Built: 1660228991
BuiltTime: Thu Aug 11 16:43:11 2022
GitCommit: ""
GoVersion: go1.18.4
Os: linux
OsArch: linux/arm64
Version: 4.2.0
Package info (e.g. output of rpm -q podman or apt list podman):
brew info podman
==> podman: stable 4.2.0 (bottled), HEAD
Tool for managing OCI containers and pods
https://podman.io/
/opt/homebrew/Cellar/podman/4.2.0 (178 files, 48MB) *
Poured from bottle on 2022-08-11 at 13:56:31
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/podman.rb
License: Apache-2.0
==> Dependencies
Build: go-md2man ✘, go@1.18 ✘
Required: qemu ✔
==> Options
--HEAD
Install HEAD version
==> Caveats
Bash completion has been installed to:
/opt/homebrew/etc/bash_completion.d
==> Analytics
install: 21,840 (30 days), 57,845 (90 days), 195,310 (365 days)
install-on-request: 21,411 (30 days), 57,072 (90 days), 194,478 (365 days)
build-error: 37 (30 days)
Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)
Yes (that is, the 4.2.0 version in brew)
Additional environment details (AWS, VirtualBox, physical, etc.): Apple M1
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 37 (29 by maintainers)
It’s a limitation in qemu-user-static-x86 binfmt support. rhel9 expects a CPU supporting the x86_64-v2 instruction set, but qemu-user-static binfmt support is configured to run
qemu-x86_64-staticdirectly (see/usr/lib/binfmt.d/qemu-x86_64-static.confssh’ing into a linux VM running on the M1 and running
ld-linux-x86_64.so.2 --helpreturns:It’s possible to use
--cpuas an argument toqemu-x86_64-staticto get something better:(the important bit is the last line)
I tried to configure binfmt to use an intermediate bash script or go program to start qemu-x86_64-static with the correct arguments, this worked when trying to start an x86_64 binary directly from the command line inside the VM, but when I then try to use
podman run, this fails:(with an intermediate go binary). With a shell script, this gives an error message about too many symlinks. I’m not quite sure what’s wrong/what’s confusing podman.
I have written a quick and dirty program to test the feature level of an x86 CPU here (it will also list which features of a feature level is supported and which are missing): https://github.com/CKingX/x86-feature-test/releases/tag/v1.0.0 This may be of interest to this issue.
We just install the
qemu-user-static-x86package. Any issues would need to be taken up withqemu(link to bugzilla can be found here: https://src.fedoraproject.org/rpms/qemu )Most likely https://github.com/tonistiigi/binfmt/blob/master/patches/cpu-max/0001-default-to-cpu-max-on-x86-and-arm.patch
Responding to question in Twitter whether this problem also occurs with Docker instead of Podman:
The issue is about how to get this correctly configured to allow
binfmt_miscandpodmanto correctly run the binaries with the changed-cpusetting.