binfmt: Intermittent segmentation fault running Node emulated with target arm64, host amd64
Hello, thanks for binfmt.
docker run --privileged --rm tonistiigi/binfmt --version
binfmt/a161c41 qemu/v7.0.0 go/1.18.5
When running
docker buildx build . --platform=arm64 --progress=plain --no-cache
on a Dockerfile which installs node on ubuntu, node --version in the container build intermittently segfaults. I captured it happening with QEMU_STRACE=1 as below.
#14 [11/13] RUN node --version
#14 0.352 1 brk(NULL) = 0x0000005500030000
#14 0.354 1 uname(0x5502830768) = 0
#14 0.355 1 faccessat(AT_FDCWD,"/etc/ld.so.preload",R_OK,0) = -1 errno=2 (No such file or directory)
#14 0.356 1 openat(AT_FDCWD,"/etc/ld.so.cache",O_RDONLY|O_CLOEXEC) = 3
#14 0.356 1 newfstatat(3,"",0x000000550282fd10,0x1000) = 0
#14 0.356 1 mmap(NULL,7675,PROT_READ,MAP_PRIVATE,3,0) = 0x0000005502870000
#14 0.356 1 close(3) = 0
#14 0.357 1 openat(AT_FDCWD,"/lib/aarch64-linux-gnu/libc.so.6",O_RDONLY|O_CLOEXEC) = 3
#14 0.357 1 read(3,0x282fec0,832) = 832
#14 0.357 1 newfstatat(3,"",0x000000550282fd50,0x1000) = 0
#14 0.358 1 mmap(NULL,8192,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) = 0x0000005502872000
#14 0.358 1 mmap(NULL,1805928,PROT_NONE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) = 0x0000005502874000
#14 0.358 1 mmap(0x0000005502880000,1740392,PROT_EXEC|PROT_READ,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIXED,3,0) = 0x0000005502880000
#14 0.358 1 munmap(0x0000005502874000,49152) = 0
#14 0.358 1 munmap(0x0000005502a29000,15976) = 0
#14 0.358 1 mprotect(0x0000005502a08000,61440,PROT_NONE) = 0
#14 0.359 1 mmap(0x0000005502a17000,24576,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIXED,3,0x187000) = 0x0000005502a17000
#14 0.359 1 mmap(0x0000005502a1d000,48744,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS|MAP_FIXED,-1,0) = 0x0000005502a1d000
#14 0.359 1 close(3) = 0
#14 0.361 1 set_tid_address(365114633616,365114609728,365114633600,365114614160,365114633856,365114633728) = 1
#14 0.361 1 set_robust_list(365114633632,24,365114633632,1,0,365114633680) = -1 errno=38 (Function not implemented)
#14 0.361 1 Unknown syscall 293
#14 0.363 1 mprotect(0x0000005502a17000,16384,PROT_READ) = 0
#14 0.364 1 mprotect(0x000000550002b000,8192,PROT_READ) = 0
#14 0.365 1 mprotect(0x000000550286b000,8192,PROT_READ) = 0
#14 0.365 1 prlimit64(0,3,0,365114362072,365115958040,40) = 0
#14 0.365 1 munmap(0x0000005502870000,7675) = 0
#14 0.366 1 getuid() = 54902
#14 0.366 1 getgid() = 54902
#14 0.366 1 getpid() = 1
#14 0.367 1 rt_sigaction(SIGCHLD,0x0000005502830888,NULL) = 0
#14 0.367 1 geteuid() = 54902
#14 0.367 1 getrandom(365116397872,8,1,365114633408,1,0) = 8
#14 0.368 1 brk(NULL) = 0x0000005500030000
#14 0.368 1 brk(0x0000005500051000) = 0x0000005500051000
#14 0.369 1 getppid() = 0
#14 0.371 1 getcwd(0x5500030390,4096) = 10
#14 0.372 1 geteuid() = 54902
#14 0.372 1 getegid() = 54902
#14 0.372 1 rt_sigaction(SIGINT,NULL,0x00000055028308f0) = 0
#14 0.372 1 rt_sigaction(SIGINT,0x0000005502830858,NULL) = 0
#14 0.372 1 rt_sigaction(SIGQUIT,NULL,0x00000055028308f0) = 0
#14 0.372 1 rt_sigaction(SIGQUIT,0x0000005502830858,NULL) = 0
#14 0.372 1 rt_sigaction(SIGTERM,NULL,0x00000055028308f0) = 0
#14 0.372 1 rt_sigaction(SIGTERM,0x0000005502830858,NULL) = 0
#14 0.375 1 newfstatat(AT_FDCWD,"/opt/node-v20.17.0-linux-arm64/bin//node",0x0000005502830838,0) = 0
#14 0.376 1 rt_sigprocmask(SIG_SETMASK,0x0000005502830838,NULL) = 0
#14 0.376 1 clone(CLONE_VM|CLONE_VFORK|0x11,child_stack=0x00000055028307e0,parent_tidptr=0x0000000000000000,tls=0x0000000000000000,child_tidptr=0xfffffffe7fffffff) = 8
#14 0.376 1 clone(CLONE_VM|CLONE_VFORK|0x11,child_stack=0x00000055028307e0,parent_tidptr=0x0000000000000000,tls=0x0000000000000000,child_tidptr=0xfffffffe7fffffff) = 0
#14 0.376 1 rt_sigprocmask(SIG_SETMASK,0x00000055028306d8,0x0000005502830758) = 0
#14 0.377 8 rt_sigprocmask(SIG_SETMASK,0x00000055028306d8,0x0000005502830758) = 0
#14 0.521 1 wait4(-1,365114361732,0,0,365114633408,0) = 8
#14 0.521 --- SIGCHLD {si_signo=SIGCHLD, si_code=3, si_pid=8, si_uid=54902, si_status=11, si_utime=0, si_stime=0} ---
#14 0.521 1 rt_sigreturn(17,365072415712,1,0,365072404480,0) = -1 errno=513 (Successful exit from sigreturn)
#14 0.525 Segmentation fault (core dumped)
#14 0.525 1 write(2,0x2830808,33) = 33
#14 0.526 1 wait4(-1,365114361732,1,0,365114633408,365114361882) = -1 errno=10 (No child process)
#14 0.529 1 exit_group(139)
#14 ERROR: process "/bin/sh -c node --version" did not complete successfully: exit code: 139
------
> [11/13] RUN node --version:
0.376 1 clone(CLONE_VM|CLONE_VFORK|0x11,child_stack=0x00000055028307e0,parent_tidptr=0x0000000000000000,tls=0x0000000000000000,child_tidptr=0xfffffffe7fffffff) = 0
0.376 1 rt_sigprocmask(SIG_SETMASK,0x00000055028306d8,0x0000005502830758) = 0
0.377 8 rt_sigprocmask(SIG_SETMASK,0x00000055028306d8,0x0000005502830758) = 0
0.521 1 wait4(-1,365114361732,0,0,365114633408,0) = 8
0.521 --- SIGCHLD {si_signo=SIGCHLD, si_code=3, si_pid=8, si_uid=54902, si_status=11, si_utime=0, si_stime=0} ---
0.521 1 rt_sigreturn(17,365072415712,1,0,365072404480,0) = -1 errno=513 (Successful exit from sigreturn)
0.525 Segmentation fault (core dumped)
0.525 1 write(2,0x2830808,33) = 33
0.526 1 wait4(-1,365114361732,1,0,365114633408,365114361882) = -1 errno=10 (No child process)
0.529 1 exit_group(139)
------
1 warning found (use docker --debug to expand):
- LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 12)
Dockerfile:35
--------------------
33 | ENV QEMU_STRACE=1
34 |
35 | >>> RUN node --version
36 | RUN npm --version
37 | RUN yarn --version
--------------------
ERROR: failed to solve: process "/bin/sh -c node --version" did not complete successfully: exit code: 139
For comparison I also provide a log with node --version working, at the same point as where the crash occurred above, but this time working, as it is an intermittent problem.
#14 [11/13] RUN node --version
#14 0.386 1 brk(NULL) = 0x0000005500030000
#14 0.389 1 uname(0x5502830768) = 0
#14 0.391 1 faccessat(AT_FDCWD,"/etc/ld.so.preload",R_OK,0) = -1 errno=2 (No such file or direc
tory)
#14 0.393 1 openat(AT_FDCWD,"/etc/ld.so.cache",O_RDONLY|O_CLOEXEC) = 3
#14 0.393 1 newfstatat(3,"",0x000000550282fd10,0x1000) = 0
#14 0.393 1 mmap(NULL,7675,PROT_READ,MAP_PRIVATE,3,0) = 0x0000005502870000
#14 0.393 1 close(3) = 0
#14 0.394 1 openat(AT_FDCWD,"/lib/aarch64-linux-gnu/libc.so.6",O_RDONLY|O_CLOEXEC) = 3
#14 0.394 1 read(3,0x282fec0,832) = 832
#14 0.395 1 newfstatat(3,"",0x000000550282fd50,0x1000) = 0
#14 0.395 1 mmap(NULL,8192,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) = 0x00000055028
72000
#14 0.396 1 mmap(NULL,1805928,PROT_NONE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) = 0x0000005502874000
#14 0.396 1 mmap(0x0000005502880000,1740392,PROT_EXEC|PROT_READ,MAP_PRIVATE|MAP_DENYWRITE|MAP_F
IXED,3,0) = 0x0000005502880000
#14 0.396 1 munmap(0x0000005502874000,49152) = 0
#14 0.396 1 munmap(0x0000005502a29000,15976) = 0
#14 0.396 1 mprotect(0x0000005502a08000,61440,PROT_NONE) = 0
#14 0.397 1 mmap(0x0000005502a17000,24576,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_DENYWRITE|MAP_FI
XED,3,0x187000) = 0x0000005502a17000
#14 0.397 1 mmap(0x0000005502a1d000,48744,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS|MAP_FI
XED,-1,0) = 0x0000005502a1d000
#14 0.398 1 close(3) = 0
#14 0.402 1 set_tid_address(365114633616,365114609728,365114633600,365114614160,365114633856,36
5114633728) = 1
#14 0.402 1 set_robust_list(365114633632,24,365114633632,1,0,365114633680) = -1 errno=38 (Funct
ion not implemented)
#14 0.402 1 Unknown syscall 293
#14 0.405 1 mprotect(0x0000005502a17000,16384,PROT_READ) = 0
#14 0.405 1 mprotect(0x000000550002b000,8192,PROT_READ) = 0
#14 0.407 1 mprotect(0x000000550286b000,8192,PROT_READ) = 0
#14 0.408 1 prlimit64(0,3,0,365114362072,365115958040,40) = 0
#14 0.409 1 munmap(0x0000005502870000,7675) = 0
#14 0.411 1 getuid() = 54902
#14 0.411 1 getgid() = 54902
#14 0.411 1 getpid() = 1
#14 0.411 1 rt_sigaction(SIGCHLD,0x0000005502830888,NULL) = 0
#14 0.412 1 geteuid() = 54902
#14 0.412 1 getrandom(365116397872,8,1,365114633408,1,0) = 8
#14 0.413 1 brk(NULL) = 0x0000005500030000
#14 0.413 1 brk(0x0000005500051000) = 0x0000005500051000
#14 0.416 1 getppid() = 0
#14 0.418 1 getcwd(0x5500030390,4096) = 10
#14 0.420 1 geteuid() = 54902
#14 0.420 1 getegid() = 54902
#14 0.420 1 rt_sigaction(SIGINT,NULL,0x00000055028308f0) = 0
#14 0.421 1 rt_sigaction(SIGINT,0x0000005502830858,NULL) = 0
#14 0.421 1 rt_sigaction(SIGQUIT,NULL,0x00000055028308f0) = 0
#14 0.421 1 rt_sigaction(SIGQUIT,0x0000005502830858,NULL) = 0
#14 0.421 1 rt_sigaction(SIGTERM,NULL,0x00000055028308f0) = 0
#14 0.421 1 rt_sigaction(SIGTERM,0x0000005502830858,NULL) = 0
#14 0.426 1 newfstatat(AT_FDCWD,"/opt/node-v20.17.0-linux-arm64/bin//node",0x0000005502830838,0
) = 0
#14 0.427 1 rt_sigprocmask(SIG_SETMASK,0x0000005502830838,NULL) = 0
#14 0.427 1 clone(CLONE_VM|CLONE_VFORK|0x11,child_stack=0x00000055028307e0,parent_tidptr=0x0000
000000000000,tls=0x0000000000000000,child_tidptr=0xfffffffe7fffffff) = 8
#14 0.428 1 clone(CLONE_VM|CLONE_VFORK|0x11,child_stack=0x00000055028307e0,parent_tidptr=0x0000
000000000000,tls=0x0000000000000000,child_tidptr=0xfffffffe7fffffff) = 0
#14 0.428 1 rt_sigprocmask(SIG_SETMASK,0x00000055028306d8,0x0000005502830758) = 0
#14 0.428 8 rt_sigprocmask(SIG_SETMASK,0x00000055028306d8,0x0000005502830758) = 0
#14 0.507 8 brk(NULL) = 0x00000000052a8000
#14 0.509 8 uname(0x5500800748) = 0
#14 0.512 8 faccessat(AT_FDCWD,"/etc/ld.so.preload",R_OK,0) = -1 errno=2 (No such file or direc
tory)
#14 0.513 8 openat(AT_FDCWD,"/etc/ld.so.cache",O_RDONLY|O_CLOEXEC) = 3
#14 0.513 8 newfstatat(3,"",0x00000055007ffcf0,0x1000) = 0
#14 0.513 8 mmap(NULL,7675,PROT_READ,MAP_PRIVATE,3,0) = 0x0000005500840000
#14 0.513 8 close(3) = 0
#14 0.514 8 openat(AT_FDCWD,"/lib/aarch64-linux-gnu/libdl.so.2",O_RDONLY|O_CLOEXEC) = 3
#14 0.514 8 read(3,0x7ffea0,832) = 832
#14 0.515 8 newfstatat(3,"",0x00000055007ffd30,0x1000) = 0
#14 0.515 8 mmap(NULL,8192,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) = 0x00000055008
42000
#14 0.516 8 mmap(NULL,135200,PROT_NONE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) = 0x0000005500844000
#14 0.516 8 mmap(0x0000005500850000,69664,PROT_EXEC|PROT_READ,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIX
ED,3,0) = 0x0000005500850000
#14 0.516 8 munmap(0x0000005500844000,49152) = 0
#14 0.517 8 munmap(0x0000005500862000,12320) = 0
#14 0.517 8 mprotect(0x0000005500851000,61440,PROT_NONE) = 0
#14 0.517 8 mmap(0x0000005500860000,8192,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIX
ED,3,0) = 0x0000005500860000
#14 0.518 8 close(3) = 0
#14 0.518 8 openat(AT_FDCWD,"/lib/aarch64-linux-gnu/libstdc++.so.6",O_RDONLY|O_CLOEXEC) = 3
#14 0.518 8 read(3,0x7ffe80,832) = 832
#14 0.518 8 newfstatat(3,"",0x00000055007ffd10,0x1000) = 0
#14 0.519 8 mmap(NULL,2332704,PROT_NONE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) = 0x0000005500866000
#14 0.519 8 mmap(0x0000005500870000,2267168,PROT_EXEC|PROT_READ,MAP_PRIVATE|MAP_DENYWRITE|MAP_F
IXED,3,0) = 0x0000005500870000
#14 0.519 8 munmap(0x0000005500866000,40960) = 0
#14 0.519 8 munmap(0x0000005500a9a000,22560) = 0
#14 0.519 8 mprotect(0x0000005500a7a000,61440,PROT_NONE) = 0
#14 0.519 8 mmap(0x0000005500a89000,57344,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_DENYWRITE|MAP_FI
XED,3,0x209000) = 0x0000005500a89000
#14 0.519 8 mmap(0x0000005500a97000,10272,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS|MAP_FI
XED,-1,0) = 0x0000005500a97000
#14 0.519 8 close(3) = 0
#14 0.519 8 openat(AT_FDCWD,"/lib/aarch64-linux-gnu/libm.so.6",O_RDONLY|O_CLOEXEC) = 3
#14 0.519 8 read(3,0x7ffe60,832) = 832
#14 0.519 8 newfstatat(3,"",0x00000055007ffcf0,0x1000) = 0
#14 0.519 8 mmap(NULL,680048,PROT_NONE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) = 0x0000005500aa0000
#14 0.519 8 mmap(0x0000005500aa0000,614512,PROT_EXEC|PROT_READ,MAP_PRIVATE|MAP_DENYWRITE|MAP_FI
XED,3,0) = 0x0000005500aa0000
#14 0.519 8 munmap(0x0000005500b37000,61552) = 0
#14 0.520 8 mprotect(0x0000005500b26000,61440,PROT_NONE) = 0
#14 0.520 8 mmap(0x0000005500b35000,8192,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIX
ED,3,0x85000) = 0x0000005500b35000
#14 0.520 8 close(3) = 0
#14 0.520 8 openat(AT_FDCWD,"/lib/aarch64-linux-gnu/libgcc_s.so.1",O_RDONLY|O_CLOEXEC) = 3
#14 0.520 8 read(3,0x7ffe40,832) = 832
#14 0.520 8 newfstatat(3,"",0x00000055007ffcd0,0x1000) = 0
#14 0.520 8 mmap(NULL,213704,PROT_NONE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) = 0x0000005500b47000
#14 0.520 8 mmap(0x0000005500b50000,148168,PROT_EXEC|PROT_READ,MAP_PRIVATE|MAP_DENYWRITE|MAP_FI
XED,3,0) = 0x0000005500b50000
#14 0.520 8 munmap(0x0000005500b47000,36864) = 0
#14 0.520 8 munmap(0x0000005500b75000,25288) = 0
#14 0.520 8 mprotect(0x0000005500b64000,61440,PROT_NONE) = 0
#14 0.520 8 mmap(0x0000005500b73000,8192,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIX
ED,3,0x13000) = 0x0000005500b73000
#14 0.520 8 close(3) = 0
#14 0.520 8 openat(AT_FDCWD,"/lib/aarch64-linux-gnu/libpthread.so.0",O_RDONLY|O_CLOEXEC) = 3
#14 0.520 8 read(3,0x7ffe20,832) = 832
#14 0.520 8 newfstatat(3,"",0x00000055007ffcb0,0x1000) = 0
#14 0.520 8 mmap(NULL,135200,PROT_NONE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) = 0x0000005500b7c000
#14 0.520 8 mmap(0x0000005500b80000,69664,PROT_EXEC|PROT_READ,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIX
ED,3,0) = 0x0000005500b80000
#14 0.520 8 munmap(0x0000005500b7c000,16384) = 0
#14 0.520 8 munmap(0x0000005500b92000,45088) = 0
#14 0.520 8 mprotect(0x0000005500b81000,61440,PROT_NONE) = 0
#14 0.520 8 mmap(0x0000005500b90000,8192,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_DENYWRITE|MAP_FIX
ED,3,0) = 0x0000005500b90000
#14 0.520 8 close(3) = 0
#14 0.520 8 openat(AT_FDCWD,"/lib/aarch64-linux-gnu/libc.so.6",O_RDONLY|O_CLOEXEC) = 3
#14 0.520 8 read(3,0x7ffe00,832) = 832
#14 0.520 8 newfstatat(3,"",0x00000055007ffc90,0x1000) = 0
#14 0.521 8 mmap(NULL,1805928,PROT_NONE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) = 0x0000005500b9e000
#14 0.521 8 mmap(0x0000005500ba0000,1740392,PROT_EXEC|PROT_READ,MAP_PRIVATE|MAP_DENYWRITE|MAP_F
IXED,3,0) = 0x0000005500ba0000
#14 0.521 8 munmap(0x0000005500b9e000,8192) = 0
#14 0.521 8 munmap(0x0000005500d49000,56936) = 0
#14 0.521 8 mprotect(0x0000005500d28000,61440,PROT_NONE) = 0
#14 0.521 8 mmap(0x0000005500d37000,24576,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_DENYWRITE|MAP_FI
XED,3,0x187000) = 0x0000005500d37000
#14 0.521 8 mmap(0x0000005500d3d000,48744,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS|MAP_FI
XED,-1,0) = 0x0000005500d3d000
#14 0.521 8 close(3) = 0
#14 0.521 8 mmap(NULL,8192,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) = 0x0000005500d
57000
#14 0.524 8 mmap(NULL,12288,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) = 0x0000005500
d59000
#14 0.524 8 set_tid_address(365086213456,365080858688,365086213440,365080863120,365086213696,36
5086213632) = 8
#14 0.524 8 set_robust_list(365086213472,24,365086213472,1,0,365086213520) = -1 errno=38 (Funct
ion not implemented)
#14 0.524 8 Unknown syscall 293
#14 0.527 8 mprotect(0x0000005500d37000,16384,PROT_READ) = 0
#14 0.527 8 mprotect(0x0000005500b90000,4096,PROT_READ) = 0
#14 0.527 8 mprotect(0x0000005500b73000,4096,PROT_READ) = 0
#14 0.527 8 mprotect(0x0000005500b35000,4096,PROT_READ) = 0
#14 0.531 8 mprotect(0x0000005500a89000,45056,PROT_READ) = 0
#14 0.531 8 mprotect(0x0000005500860000,4096,PROT_READ) = 0
#14 0.533 8 mprotect(0x000000000523c000,16384,PROT_READ) = 0
#14 0.535 8 mprotect(0x000000550083b000,8192,PROT_READ) = 0
#14 0.535 8 prlimit64(0,3,0,365080611000,365085680408,320) = 0
#14 0.536 8 munmap(0x0000005500840000,7675) = 0
#14 0.538 8 getrandom(365086120240,8,1,365086213248,1,0) = 8
#14 0.539 8 brk(NULL) = 0x00000000052a8000
#14 0.539 8 brk(0x00000000052c9000) = 0x00000000052c9000
#14 0.543 8 futex(0x0000005500a977a4,FUTEX_PRIVATE_FLAG|FUTEX_WAKE,2147483647,NULL,NULL,0) = 0
#14 0.573 8 brk(0x00000000052ea000) = 0x00000000052ea000
#14 0.582 8 clock_gettime(CLOCK_MONOTONIC,0x0000005500800c70) = 0 ({tv_sec = 13496,tv_nsec = 99
488400})
#14 0.582 8 gettimeofday(0x0000005500800c50,NULL) = 0 ({tv_sec = 1737120023,tv_usec = 146922},N
ULL)
#14 0.585 8 futex(0x000000000528ed68,FUTEX_PRIVATE_FLAG|FUTEX_WAKE,2147483647,NULL,0x0000000000
000005,5) = 0
#14 0.606 8 brk(0x000000000530b000) = 0x000000000530b000
#14 0.622 8 rt_sigprocmask(SIG_SETMASK,0x0000005500800c30,0x0000005500800b10) = 0
#14 0.622 8 rt_sigaction(SIGILL,0x00000055008009b8,0x0000005500800a50) = 0
#14 0.623 8 rt_sigaction(SIGILL,0x00000055008009b8,NULL) = 0
#14 0.623 8 rt_sigprocmask(SIG_SETMASK,0x0000005500800b10,NULL) = 0
#14 0.640 8 getpid() = 8
#14 0.640 8 capget(0x55008008a0,0x55008008a8) = 0
#14 0.640 8 getuid() = 54902
#14 0.640 8 geteuid() = 54902
#14 0.640 8 getgid() = 54902
#14 0.640 8 getegid() = 54902
#14 0.642 8 fstat(0,0x000000000527e778) = 0
#14 0.642 8 fstat(1,0x000000000527e840) = 0
#14 0.642 8 fstat(2,0x000000000527e908) = 0
#14 0.642 8 rt_sigprocmask(SIG_SETMASK,0x0000005500800a80,NULL) = 0
#14 0.642 8 rt_sigaction(SIGHUP,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(SIGINT,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(SIGQUIT,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(SIGILL,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(5,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(SIGABRT,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(7,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(SIGFPE,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(SIGUSR1,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(SIGSEGV,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(SIGUSR2,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(SIGPIPE,0x00000055008006a8,NULL) = 0
#14 0.642 8 rt_sigaction(SIGALRM,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(SIGTERM,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(16,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(SIGCHLD,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(SIGCONT,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(20,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(SIGTTIN,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(SIGTTOU,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(23,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(24,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(25,0x00000055008006a8,NULL) = 0
#14 0.642 8 rt_sigaction(26,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(27,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(28,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(29,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(30,NULL,0x0000005500800740) = 0
#14 0.642 8 rt_sigaction(31,NULL,0x0000005500800740) = 0
#14 0.643 8 fcntl(0,F_GETFL) = 0
#14 0.643 8 ioctl(0,TCGETS,0x0000005500800770) = -1 errno=25 (Not a tty)
#14 0.643 8 fstat(0,0x0000005500800830) = 0
#14 0.643 8 fcntl(1,F_GETFL) = 1
#14 0.643 8 ioctl(1,TCGETS,0x0000005500800770) = -1 errno=25 (Not a tty)
#14 0.643 8 fstat(1,0x0000005500800830) = 0
#14 0.643 8 fcntl(2,F_GETFL) = 1
#14 0.643 8 ioctl(2,TCGETS,0x0000005500800770) = -1 errno=25 (Not a tty)
#14 0.643 8 fstat(2,0x0000005500800830) = 0
#14 0.643 8 rt_sigaction(SIGINT,0x00000055008007f8,NULL) = 0
#14 0.643 8 rt_sigaction(SIGTERM,0x00000055008007f8,NULL) = 0
#14 0.643 8 prlimit64(0,7,0,365080611456,0,2147483648) = 0
#14 0.644 8 prlimit64(0,7,365080611456,0,0,2147483648) = 0
#14 0.644 8 clock_gettime(CLOCK_MONOTONIC,0x00000055008007e0) = 0 ({tv_sec = 13496,tv_nsec = 16
1099728})
#14 0.645 8 fcntl(0,F_SETFD,1) = 0
#14 0.645 8 fcntl(1,F_SETFD,1) = 0
#14 0.645 8 fcntl(2,F_SETFD,1) = 0
#14 0.645 8 fcntl(3,F_SETFD,1) = -1 errno=9 (Bad file descriptor)
#14 0.645 8 fcntl(4,F_SETFD,1) = -1 errno=9 (Bad file descriptor)
#14 0.645 8 fcntl(5,F_SETFD,1) = -1 errno=9 (Bad file descriptor)
#14 0.645 8 fcntl(6,F_SETFD,1) = -1 errno=9 (Bad file descriptor)
#14 0.645 8 fcntl(7,F_SETFD,1) = -1 errno=9 (Bad file descriptor)
#14 0.645 8 fcntl(8,F_SETFD,1) = -1 errno=9 (Bad file descriptor)
#14 0.645 8 fcntl(9,F_SETFD,1) = -1 errno=9 (Bad file descriptor)
#14 0.645 8 fcntl(10,F_SETFD,1) = -1 errno=9 (Bad file descriptor)
#14 0.645 8 fcntl(11,F_SETFD,1) = -1 errno=9 (Bad file descriptor)
#14 0.645 8 fcntl(12,F_SETFD,1) = -1 errno=9 (Bad file descriptor)
#14 0.645 8 fcntl(13,F_SETFD,1) = -1 errno=9 (Bad file descriptor)
#14 0.645 8 fcntl(14,F_SETFD,1) = -1 errno=9 (Bad file descriptor)
#14 0.645 8 fcntl(15,F_SETFD,1) = -1 errno=9 (Bad file descriptor)
#14 0.645 8 fcntl(16,F_SETFD,1) = -1 errno=9 (Bad file descriptor)
#14 0.651 8 getpid() = 8
#14 0.651 8 capget(0x55008006c0,0x55008006c8) = 0
#14 0.651 8 getuid() = 54902
#14 0.651 8 geteuid() = 54902
#14 0.651 8 getgid() = 54902
#14 0.651 8 getegid() = 54902
#14 0.651 8 getpid() = 8
#14 0.651 8 capget(0x55008006c0,0x55008006c8) = 0
#14 0.651 8 getuid() = 54902
#14 0.651 8 geteuid() = 54902
#14 0.651 8 getgid() = 54902
#14 0.651 8 getegid() = 54902
#14 0.651 8 getpid() = 8
#14 0.651 8 capget(0x55008006c0,0x55008006c8) = 0
#14 0.651 8 getuid() = 54902
#14 0.651 8 geteuid() = 54902
#14 0.651 8 getgid() = 54902
#14 0.651 8 getegid() = 54902
#14 0.652 8 getpid() = 8
#14 0.652 8 capget(0x55008006c0,0x55008006c8) = 0
#14 0.652 8 getuid() = 54902
#14 0.652 8 geteuid() = 54902
#14 0.652 8 getgid() = 54902
#14 0.652 8 getegid() = 54902
#14 0.652 8 getpid() = 8
#14 0.652 8 capget(0x55008007c0,0x55008007c8) = 0
#14 0.652 8 getuid() = 54902
#14 0.652 8 geteuid() = 54902
#14 0.652 8 getgid() = 54902
#14 0.652 8 getegid() = 54902
#14 0.658 8 getpid() = 8
#14 0.658 8 capget(0x55008007c0,0x55008007c8) = 0
#14 0.658 8 getuid() = 54902
#14 0.658 8 geteuid() = 54902
#14 0.658 8 getgid() = 54902
#14 0.658 8 getegid() = 54902
#14 0.658 8 futex(0x0000000005294960,FUTEX_PRIVATE_FLAG|FUTEX_WAKE,2147483647,NULL,NULL,0) = 0
#14 0.667 8 write(1,0x247a5d8,8) = 8
#14 0.667 v20.17.0
#14 0.667 8 write(1,0xd3c65b,1) = 1
#14 0.668 8 fstat(0,0x0000005500800a70) = 0
#14 0.668 8 fcntl(0,F_GETFL) = 0
#14 0.668 8 fstat(1,0x0000005500800a70) = 0
#14 0.668 8 fcntl(1,F_GETFL) = 1
#14 0.668 8 fstat(2,0x0000005500800a70) = 0
#14 0.668 8 fcntl(2,F_GETFL) = 1
#14 0.681 8 exit_group(0)
#14 0.686 1 wait4(-1,365114361732,0,0,365114633408,0) = 8
#14 0.686 --- SIGCHLD {si_signo=SIGCHLD, si_code=1, si_pid=8, si_uid=54902, si_status=0, si_uti
me=23, si_stime=2} ---
#14 0.686 1 rt_sigreturn(17,365072415712,1,0,365072404480,0) = -1 errno=513 (Successful exit fr
om sigreturn)
#14 0.686 1 wait4(-1,365114361732,1,0,365114633408,0) = -1 errno=10 (No child process)
#14 0.688 1 exit_group(0)
#14 DONE 0.8s
Systemd coredump:
COREDUMP_UID=54902
COREDUMP_GID=54902
COREDUMP_SIGNAL_NAME=SIGSEGV
COREDUMP_SIGNAL=11
COREDUMP_RLIMIT=18446744073709551615
COREDUMP_HOSTNAME=buildkitsandbox
COREDUMP_COMM=node
COREDUMP_EXE=/usr/bin/qemu-aarch64
COREDUMP_SLICE=system.slice
COREDUMP_PROC_LIMITS=Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size unlimited unlimited bytes
Max resident set unlimited unlimited bytes
Max processes unlimited unlimited processes
Max open files 1024 524288 files
Max locked memory 8388608 8388608 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 61107 61107 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
COREDUMP_CWD=/home/git
COREDUMP_ROOT=/
MESSAGE_ID=fc2e22bc6ee647b6b90729ab34a250b1
PRIORITY=2
COREDUMP_PACKAGE_JSON={"elfType":"coredump","elfArchitecture":"AMD x86-64"}
CODE_FILE=../systemd/src/coredump/coredump.c
CODE_LINE=931
CODE_FUNC=submit_coredump
SYSLOG_IDENTIFIER=systemd-coredump
_TRANSPORT=journal
_UID=54902
_GID=54902
_COMM=systemd-coredum
_EXE=/usr/lib/systemd/systemd-coredump
_CMDLINE=/usr/lib/systemd/systemd-coredump
_CAP_EFFECTIVE=0
_SYSTEMD_SLICE=system-systemd\x2dcoredump.slice
_MACHINE_ID=42c50f3f406e4854a6fc41b4d90bc6e2
_HOSTNAME=archlinux
_RUNTIME_SCOPE=system
COREDUMP_PID=446520
COREDUMP_TIMESTAMP=1737135759000000
COREDUMP_CMDLINE=/usr/bin/qemu-aarch64 /opt/node-v20.17.0-linux-arm64/bin//node node --version
COREDUMP_CGROUP=/system.slice/system.slice:docker:xyykonphxikhnir01ewmnw2wv
COREDUMP_OPEN_FDS=0:pipe:[725533]
pos: 0
flags: 00
mnt_id: 16
ino: 725533
1:pipe:[725534]
pos: 0
flags: 01
mnt_id: 16
ino: 725534
2:pipe:[725535]
pos: 0
flags: 01
mnt_id: 16
ino: 725535
3:/opt/node-v20.17.0-linux-arm64/bin/node
pos: 1024
flags: 0100040
mnt_id: 390
ino: 14204964
COREDUMP_PROC_STATUS=Name: node
Umask: 0022
State: S (sleeping)
Tgid: 446520
Ngid: 0
Pid: 446520
PPid: 446456
TracerPid: 0
Uid: 54902 54902 54902 54902
Gid: 54902 54902 54902 54902
FDSize: 64
Groups: 54902
NStgid: 446520 8
NSpid: 446520 8
NSpgid: 446456 1
NSsid: 446456 1
Kthread: 0
VmPeak: 251248 kB
VmSize: 251248 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 3276 kB
VmRSS: 3276 kB
RssAnon: 1024 kB
RssFile: 2252 kB
RssShmem: 0 kB
VmData: 132800 kB
VmStk: 132 kB
VmExe: 3452 kB
VmLib: 8 kB
VmPTE: 44 kB
VmSwap: 0 kB
HugetlbPages: 0 kB
CoreDumping: 1
THP_enabled: 1
untag_mask: 0xffffffffffffffff
Threads: 2
SigQ: 0/61107
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000000000000
SigCgt: 0000000000000000
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
CapBnd: 00000000a80425fb
CapAmb: 0000000000000000
NoNewPrivs: 0
Seccomp: 2
Seccomp_filters: 1
Speculation_Store_Bypass: thread vulnerable
SpeculationIndirectBranch: conditional enabled
Cpus_allowed: ff
Cpus_allowed_list: 0-7
Mems_allowed: 00000001
Mems_allowed_list: 0
voluntary_ctxt_switches: 4
nonvoluntary_ctxt_switches: 0
x86_Thread_features:
x86_Thread_features_locked:
COREDUMP_PROC_MAPS=00200000-0037b000 r--p 00000000 00:42 11177592 /usr/bin/qemu-aarch64
0037b000-006da000 r-xp 0017a000 00:42 11177592 /usr/bin/qemu-aarch64
006da000-0073d000 rw-p 004d8000 00:42 11177592 /usr/bin/qemu-aarch64
0073d000-00749000 rw-p 0053a000 00:42 11177592 /usr/bin/qemu-aarch64
00749000-0076b000 rw-p 00000000 00:00 0
00b6b000-07a13000 ---p 00000000 00:00 0 [heap]
74aa4c800000-74aa547ff000 rwxp 00000000 00:00 0
74aa547ff000-74aa54800000 ---p 00000000 00:00 0
74aa54894000-74aa548aa000 rw-p 00000000 00:00 0
74aa548ab000-74aa5498a000 rw-p 00000000 00:00 0
74aa5498a000-74aa5498c000 ---p 00000000 00:00 0
74aa5498c000-74aa549b7000 rw-p 00000000 00:00 0
74aa549b7000-74aa549bb000 r--p 00000000 00:00 0 [vvar]
74aa549bb000-74aa549bd000 r-xp 00000000 00:00 0 [vdso]
7ffd75d41000-7ffd75d62000 rw-p 00000000 00:00 0 [stack]
ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0 [vsyscall]
COREDUMP_PROC_CGROUP=0::/system.slice/system.slice:docker:xyykonphxikhnir01ewmnw2wv
COREDUMP_PROC_MOUNTINFO=390 322 0:68 / / rw,relatime - overlay overlay rw,lowerdir=/var/lib/docker/overlay2/l/HSTNAFFQXRES6QOB23>
392 390 0:79 / /proc rw,nosuid,nodev,noexec,relatime - proc proc rw
393 390 0:80 / /dev rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755,inode64
394 393 0:81 / /dev/pts rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=666
395 393 0:82 / /dev/shm rw,nosuid,nodev,noexec,relatime - tmpfs shm rw,size=65536k,inode64
396 393 0:77 / /dev/mqueue rw,nosuid,nodev,noexec,relatime - mqueue mqueue rw
397 390 0:83 / /sys ro,nosuid,nodev,noexec,relatime - sysfs sysfs ro
398 390 259:2 /var/lib/docker/buildkit/executor/resolv.conf /etc/resolv.conf ro,nosuid,nodev,noexec,rela>
399 390 259:2 /var/lib/docker/buildkit/executor/hosts.5arcfkpmp4egnbjezv1mij6uz /etc/hosts ro,nosuid,nod>
400 397 0:28 /system.slice/system.slice:docker:xyykonphxikhnir01ewmnw2wv /sys/fs/cgroup ro,nosuid,nodev,>
323 392 0:79 /bus /proc/bus ro,nosuid,nodev,noexec,relatime - proc proc rw
324 392 0:79 /fs /proc/fs ro,nosuid,nodev,noexec,relatime - proc proc rw
325 392 0:79 /irq /proc/irq ro,nosuid,nodev,noexec,relatime - proc proc rw
354 392 0:79 /sys /proc/sys ro,nosuid,nodev,noexec,relatime - proc proc rw
355 392 0:79 /sysrq-trigger /proc/sysrq-trigger ro,nosuid,nodev,noexec,relatime - proc proc rw
356 392 0:84 / /proc/acpi ro,relatime - tmpfs tmpfs ro,inode64
357 392 0:85 / /proc/asound ro,relatime - tmpfs tmpfs ro,inode64
358 392 0:80 /null /proc/kcore rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755,inode64
362 392 0:80 /null /proc/keys rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755,inode64
363 392 0:80 /null /proc/timer_list rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755,inode64
364 397 0:86 / /sys/firmware ro,relatime - tmpfs tmpfs ro,inode64
365 397 0:87 / /sys/devices/virtual/powercap ro,relatime - tmpfs tmpfs ro,inode64
366 392 0:88 / /proc/scsi ro,relatime - tmpfs tmpfs ro,inode64
COREDUMP_PROC_AUXV=[384B blob data]
COREDUMP_CONTAINER_CMDLINE=/usr/bin/qemu-aarch64 /bin/sh /bin/sh -c $'node --version'
COREDUMP_ENVIRON=HOME=/home/git
GIT_HOME_DIR=/home/git
QEMU_STRACE=1
PATH=/opt/node-v20.17.0-linux-arm64/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
CACHE_BUST=1
PWD=/home/git
COREDUMP_FILENAME=/var/lib/systemd/coredump/core.node.54902.3aa96f3b05b243ae83d4b1841b85da42.446520.1737135759000000.zst
MESSAGE=Process 446520 (node) of user 54902 dumped core.
Module /usr/bin/qemu-aarch64 (deleted) without build-id.
Stack trace of thread 8:
#0 0x00000000006c91df n/a (/usr/bin/qemu-aarch64 (deleted) + 0x4c91df)
ELF object binary architecture: AMD x86-64
Dockerfile:
FROM ubuntu:22.04
ENV GIT_HOME_DIR=/home/git
RUN apt-get update && apt-get -y install wget --no-install-recommends
RUN wget --no-check-certificate --quiet -O node.tar.gz \
https://nodejs.org/download/release/v20.17.0/node-v20.17.0-linux-arm64.tar.gz
RUN tar -xf node.tar.gz --directory=/opt/
ENV PATH "/opt/node-v20.17.0-linux-arm64/bin/:$PATH"
RUN npm install -g npm@10.8.2
RUN apt-get update \
&& apt-get -y install build-essential make python3-minimal --no-install-recommends \
&& apt-get clean
RUN npm install -g yarn@^1.22
RUN groupadd --gid 54902 git \
&& useradd --comment "Jenkins User" --home-dir ${GIT_HOME_DIR} --uid 54902 --gid 54902 --create-home git
USER git
WORKDIR ${GIT_HOME_DIR}
ARG CACHE_BUST=1
RUN echo $CACHE_BUST
ENV QEMU_STRACE=1
RUN node --version
RUN npm --version
RUN yarn --version
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 12
- Comments: 70 (16 by maintainers)
Commits related to this issue
- Explicitly use QEMU v8 for server image builds This works around the compilation issues observed with QEMU v7 (which is still used by docker/setup-qemu-action by default) as described in https://gith... — committed to mserajnik/vmangos-deploy by mserajnik 2 years ago
- Explicitly use QEMU 8.x for server image builds This works around the compilation issues observed with QEMU 7.x (which is still used by docker/setup-qemu-action by default) as described in https://gi... — committed to mserajnik/vmangos-deploy by mserajnik 2 years ago
- ci: limit qemu-user-static to arm64 By limiting it to the only used platform we hopefully can work around issues where files magically matched the ppc64 handler. Example output from dmesg: segfault... — committed to fmoessbauer/kas by fmoessbauer 2 years ago
- ci: replace copy deployment of binfmt with distro version This fixes a qemu build issue observed on the (non versioned) tonistiigi/binfmt:latest@sha256:f6b82a01e1... qemu-user-static deploy image. E... — committed to fmoessbauer/kas by fmoessbauer 2 years ago
- ci: Replace binfmt container with Debian qemu-user-static This apparently fixes sporadic crashes of arm64 image builds, see also [1] and [2]. Ubuntu's version of qemu-user does not seem to have this ... — committed to siemens/kas by jan-kiszka 2 years ago
- ci: Replace binfmt container with Debian qemu-user-static This apparently fixes sporadic crashes of arm64 image builds, see also [1] and [2]. Ubuntu's version of qemu-user does not seem to have this ... — committed to siemens/kas by jan-kiszka 2 years ago
- CI: bump qemu to v8 - see: https://github.com/tonistiigi/binfmt/issues/215 — committed to KFERMercer/chaturbate-grabber by KFERMercer 2 years ago
- CI: bump QEMU to v8.1.5 - see: https://github.com/tonistiigi/binfmt/issues/215 — committed to KFERMercer/chaturbate-grabber by KFERMercer 2 years ago
- ci(fix): use `tonistiigi/binfmt:qemu-v8.1.5` image for qemu xref https://github.com/docker/setup-qemu-action/issues/188 xref https://github.com/tonistiigi/binfmt/issues/215 — committed to mayeut/manylinux-ubuntu by mayeut 2 years ago
- ci: Replace binfmt container with Debian qemu-user-static This apparently fixes sporadic crashes of arm64 image builds, see also [1] and [2]. Ubuntu's version of qemu-user does not seem to have this ... — committed to fmoessbauer/kas by jan-kiszka 2 years ago
- https://github.com/tonistiigi/binfmt/issues/215 — committed to lrybak/hassio-owserver by lrybak 2 years ago
- ci: Replace binfmt container with Debian qemu-user-static This apparently fixes sporadic crashes of arm64 image builds, see also [1] and [2]. Ubuntu's version of qemu-user does not seem to have this ... — committed to fmoessbauer/kas by jan-kiszka 2 years ago
- Use qemu v8 to prevent qemu segfault during unit tests. See https://github.com/tonistiigi/binfmt/issues/215. — committed to jlesage/docker-baseimage by jlesage 2 years ago
- Use qemu v8 to prevent qemu segfault during unit tests. See https://github.com/tonistiigi/binfmt/issues/215. — committed to jlesage/docker-baseimage-gui by jlesage 2 years ago
- use v8 tonistiigi/binfmt As seen in: https://github.com/docker/build-push-action/issues/1309#issuecomment-2618650540 https://github.com/tonistiigi/binfmt/issues/215#issuecomment-2611042596 — committed to kahara/vushf-exporter by kahara 2 years ago
- use v8 tonistiigi/binfmt (#35) * cache image for docker/setup-qemu-action * use v8 tonistiigi/binfmt As seen in: https://github.com/docker/build-push-action/issues/1309#issuecomment-26186505... — committed to kahara/vushf-exporter by kahara 2 years ago
- Update qemu to fix aarch64 wheel action (#1056) ### Before submitting Please complete the following checklist when submitting a PR: - [ ] All new features must include a unit test. If you've f... — committed to PennyLaneAI/pennylane-lightning by josephleekl 2 years ago
- ci: Replace binfmt container with Debian qemu-user-static This apparently fixes sporadic crashes of arm64 image builds, see also [1] and [2]. Ubuntu's version of qemu-user does not seem to have this ... — committed to siemens/kas by jan-kiszka 2 years ago
- ci: Replace binfmt container with Debian qemu-user-static This apparently fixes sporadic crashes of arm64 image builds, see also [1] and [2]. Ubuntu's version of qemu-user does not seem to have this ... — committed to fmoessbauer/kas by jan-kiszka 2 years ago
- Build docker images with ubuntu-22.04 This is a workaround for an upstream issue with QEMU and binfmt on 24.04 https://github.com/tonistiigi/binfmt/issues/215 — committed to paul121/farmOS by paul121 2 years ago
@ajbarber After some digging I’m pretty sure this issue relates to a kernel hardening. This also explains why various qemu versions are affected. More details can be found in this Debian bug: [1]. This bug first happened after [2] was applied (which later was reverted) and reverted again [3] after a fix for QEMU in Debian was available. Probably Ubuntu included just the kernel patch (revert-revert) but not the QEMU patch which then broke things again.
I’ve also started hitting segmentation faults today when building a multi-platform Docker image on GitHub Actions with a
ubuntu-24.04runner targetinglinux/amd64andlinux/arm64(using a commondocker/setup-qemu-action@v3anddocker/setup-buildx-action@v3setup) and found this issue report through https://github.com/docker/setup-qemu-action/issues/188 where a similar problem is described.In my case, the segmentation faults occur when compiling the CPP application (that is deployed via the Docker image) during the build of the
linux/arm64image (so when QEMU is used):The error occurs at random steps in the compilation process. I’ve had no issues building this Docker image hundreds of times over the past year until today; the last successful build with the same code revision before the segmentation faults started happening was at 00:00 GMT today (23.01.2025), so it looks like something must’ve changed/been updated since then (or I somehow got lucky before and didn’t hit the issue; that seems a bit unlikely though, considering I’ve had a segmentation fault in every single one of the six workflow runs I did today).
Edit: Using QEMU 8.1.5 (as suggested by @smoke) instead of 7.0.0 (which is currently still tagged as
latest) seems to work for me:Edit 2: Using QEMU 9.2.0 also works fine for me:
tonistiigi/binfmt:latestwith QEMU 9.2.0 has been pushed: https://github.com/tonistiigi/binfmt/releases/tag/deploy%2Fv9.2.0-51@ajbarber I suggest closing this issue. Most people used
tonistiigi/binfmt:latest[^1] and ran into issues due to theubuntu-24.04GitHub Runner.[^1]:
tonistiigi/binfmt:latestcurrently (Feb 8, 2025) is (and has been since Aug 2, 2022)tonistiigi/binfmt:qemu-v7.0.0.The reason for the real issue is described in https://github.com/tonistiigi/binfmt/issues/215#issuecomment-2613004741.
🤐
We can add the patches to https://github.com/tonistiigi/binfmt/tree/master/patches if you want to test master build.
@benz0li Ubuntu is not the only distro that affected by this. I have the same issue using Debian and Fedora with
tonistiigi/binfmt:latestuntil I switched toqemu-user-staticdistro package.Since old qemu versions are now broken due to upstream kernel hardening, I would recommend keeping this issue open until the
latestdocker tag gets updated.I’m also encountering this. I’m using the github qemu action to build a variety of ruby versions for different architectures. Since the last 3 days,
arm64-v8,ppc64leands390xstarted to fail with pretty cryptic and random errors, both during configuring and building:Some of the errors I encountered
I run my actions on ubuntu-24.04, and coincidentally there was a runner image update right when this started. I compared builds and with
20250105.1.0they passed, while with20250120.5.0they failed. So, I find it likely that something that was updated in this changelog is responsible: https://github.com/actions/runner-images/releases/tag/ubuntu24%2F20250120.5.It updates buildx from
0.19.3to0.20.0, maybe it is their bug? I have no idea how to verify. The other software updates seem benign. On the other hand, this issue preceeds the buildx release by a few days so these may be two separate things.UsingNevermind, I got another build failure. But it looks to be much more stabletonistiigi/binfmt:qemu-v8.1.5seems to solve it for me as well.Hitting similar issues when compiling php extensions while building a base image on top of
php:8.3-fpm-bookwormneither v7.0.0 nor 8.1.5 are working in this case.
I’ve got a similar thing with regular segfaults in gcc while building C or Go applications. It appears that when I create an image with a more recent version of qemu (8.1.5) which is the latest in the repo and use that to install the emulators, the segfault does not happen.
Is there a reason that the later versions (later than v7) have not been pushed to dockerhub as the latest version?
The fix seems to be included in version 9.2.2 @tonistiigi
@crazy-max same for me, 9.2 doesn’t solve the segmentation fault (reproducible in 4/4 runs with Ubuntu 22.4 and 24.4). Also had to revert to version 7.0.0-28. Please reopen the issue, as this error affects tens of thousands of people using the qemu GitHub action.
Thanks very much @fmoessbauer. Reading some of those materials you linked, the segfaulting in question of qemu depends not only on release versions, but also the configuration flags passed at build time.
We have maintainers of qemu saying clearly not to configure with
--disable-pieor you will get crashes here. But it appears that is precisely what this repo is doing:https://github.com/tonistiigi/binfmt/blob/85908cc7576a423dcf514eba5d05df8547c73513/scripts/configure_qemu.sh#L65
To confirm @fmoessbauer’s hypothesis, I also replicated the crash/no crash behaviour of
qemuprovided by this repo by turning off the aslr kernel parametersysctl kernel.randomize_va_space=0(off) and 2 (on). 2 leads to a crash as my original issue, 0 does not. If I put on the ubuntu 22.04 shipped qemu on the container instead, which isqemu-aarch64 version 6.2.0 (Debian 1:6.2+dfsg-2ubuntu6.24), I can’t replicate the crash with aslr on or off. So in ubuntu’s packaging build scripts I’m guessing they have removed the--disable-pie, but I have not confirmed this.I think qemu was forward patched in 8.1 to deal with things either way: https://gitlab.com/qemu-project/qemu/-/issues/1763#note_1508827541
So we need either to remove the line above in
configure_qemu.shor [edited for clarity] bump latest tag to qemu > 8.1.@tonistiigi do you accept PRs?
I have stumbled on similar thing in different context and ecosystem https://github.com/docker/setup-qemu-action/issues/188
We have a Ruby app depends on quite a bunch of “native” extensions packages - that are packages having to be built through gcc
Targeting
linux/amd64,linux/arm64we have consistent
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.caused bygcc: internal compiler error: Segmentation fault signal terminated program cc1but that will affect anything built by gcc or similar (nodejs packages, etc.).Similar thing happened with a Go project needed some stuff built with gcc.
I have also managed to reproduce it locally with qemu v7.
What worked was updating to qemu v8, give it a try if it will help.
I’ve experienced it on 22.04 again, then i just switched to 24.04 in my GHA workflow and it started working again, so still a bit hit or miss, but depedent on the host machine either way it appears.
In both cases the latest image with QEMU 9.2.2 was pulled (and presumably used).
@tonistiigi @Earlopain
This worked for awhile, but started failing again with the exact same error… I was able to work around it as before with the following, but is it possible that 9.2.2-52 may have reintroduced the issue and a subsequent release resolved it again?
Do additional tests need to run before Docker Hub
latesttag can be applied to the latest image?Wow that did the trick. Thank you so much @bmhowe23.
@jaywonchung I think you likely need to put
docker/setup-qemu-actionbeforedocker/setup-buildx-actionin order for the binfmt version callout to take effect.No, I made a PR to update qemu to 9.2.2 like suggested here. I was waiting for my builds to finish and with that qemu version everything is green for me as well.
Hopefully this works for others too and when the latest tag its switched over it’s finally fixed
The current theory is the kernel bump lead the runners into this issue https://gitlab.com/qemu-project/qemu/-/issues/1913 which was fixed only recently in version 9.2.2 which the container used with latest does not run yet.
He is talking about the docker/setup-qemu-action repository but it is not a solution.
Could be related to https://gitlab.com/qemu-project/qemu/-/issues/1913, fix was merged into master yesterday.
9.2.0-51 didn’t fix the issue for me
7.0.0-28 works as expected.
Unfortunately, just like @cagnulein reported, v9.2.0 also doesn’t help to fix the segmentation fault while trying to install
libc-binwith Ubuntu 24.04.1 as base layer.QEMU_STRACE
A similiar problem has already been discussed in this issue: https://github.com/docker/buildx/issues/1170, but changing the whole build command to
didn’t help either. The build then starts again to segfault at arbritrary points in time:
I tried a lot of different combinations, but the build either crashes when using
gccor when installinglibc-bin. To be honest, I’m very confused and wondering, if tonistiigi/binfmt:qemu-v9.2.0 and multiarch/qemu-user-static should even be used together.Just tried it and
tonistiigi/binfmt:qemu-v9.2.0works without issues for my setup, liketonistiigi/binfmt:qemu-v8.1.5does as well.Same issue unfortunately
Setting up qtbase5-private-dev:arm64 (5.15.2+dfsg-9+deb11u1) ... Setting up libqt5texttospeech5-dev:arm64 (5.15.2-2) ... Setting up qtpositioning5-dev:arm64 (5.15.2+dfsg-2) ... Setting up qtlocation5-dev:arm64 (5.15.2+dfsg-2) ... Processing triggers for libc-bin (2.31-13+deb11u11) ... qemu: uncaught target signal 11 (Segmentation fault) - core dumped Segmentation fault (core dumped) qemu: uncaught target signal 11 (Segmentation fault) - core dumped Segmentation fault (core dumped) dpkg: error processing package libc-bin (--configure): installed libc-bin package post-installation script subprocess returned error exit status 139tonistiigi/binfmt:masteris now on top of QEMU 9.2.0 if you want to test:@ajbarber From
tonistiigi/binfmt:latestwhich I wrongly assumed it points to the latest build, but no it’s a 2 years old image…@cagnulein in my case the GH runner image is
ubuntu-24.04 v20250120.5.0, the buildx multi-arch CGO build used to fail constantly, but withtonistiigi/binfmt:qemu-v8.1.5-43works fine, 6 builds and so far no more seg fault.Same issue with Ubuntu 22-04 unfortunately
docker/setup-qemu-action uses
tonistiigi/binfmt:latestby default. ℹ This can be customised.When you say locally, do you mean outside of a container, just on the CPU?
I’ve just tried to reproduce with simply
qemu-aarch64v7.2.15, and an ARM64 node binary on my AMD64 metal (no docker) and I can’t replicate the segfault.It could be a
qemu-aarch64user binary issue, or an issue with a c/c++ runtime library on the target container. One way to eliminate the container Linux C/C++ libraries being the cause is to update the Linux version running in docker and see if the problem persists.(Edited, realised I’m running into this on both ubuntu 20.04 and 22.04)
Thanks @smoke. This repo only supports up to v7 though right? In the meantime I will try and get a stack, I can see it has been removed in the core dump logout above.