incus: Incus 6.11 breaks Nvidia Compute Runtime

After updating to the latest version of Incus 6.11, I can no longer start any of my containers that have the following in its config or part of a profile:

config: nvidia.driver.capabilities: all nvidia.require.cuda: “true” nvidia.runtime: “true”

incus start ai Error: Failed to run: /opt/incus/bin/incusd forkstart ai /var/lib/incus/containers /run/incus/ai/lxc.conf: exit status 1 Try incus info --show-log ai for more info

incus info --show-log ai Name: ai Description: Status: STOPPED Type: container Architecture: x86_64 Created: 2025/02/22 02:19 PST Last Used: 2025/04/04 19:08 PDT

Log:

lxc ai 20250405020851.656 ERROR utils - …/src/lxc/utils.c:run_buffer:571 - Script exited with status 1 lxc ai 20250405020851.656 ERROR conf - …/src/lxc/conf.c:lxc_setup:3948 - Failed to run mount hooks lxc ai 20250405020851.656 ERROR start - …/src/lxc/start.c:do_start:1268 - Failed to setup container “ai” lxc ai 20250405020851.656 ERROR sync - …/src/lxc/sync.c:sync_wait:34 - An error occurred in another process (expected sequence number 4) lxc ai 20250405020851.662 WARN network - …/src/lxc/network.c:lxc_delete_network_priv:3674 - Failed to rename interface with index 0 from “eth0” to its initial name “veth129a886f” lxc ai 20250405020851.662 ERROR start - …/src/lxc/start.c:__lxc_start:2114 - Failed to spawn container “ai” lxc ai 20250405020851.662 WARN start - …/src/lxc/start.c:lxc_abort:1037 - No such process - Failed to send SIGKILL via pidfd 17 for process 6716 lxc ai 20250405020851.662 ERROR lxccontainer - …/src/lxc/lxccontainer.c:wait_on_daemonized_start:837 - Received container state “ABORTING” instead of “RUNNING” lxc 20250405020851.750 ERROR af_unix - …/src/lxc/af_unix.c:lxc_abstract_unix_recv_fds_iov:218 - Connection reset by peer - Failed to receive response lxc 20250405020851.750 ERROR commands - …/src/lxc/commands.c:lxc_cmd_rsp_recv_fds:128 - Failed to receive file descriptors for command “get_init_pid”

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 2
  • Comments: 24 (10 by maintainers)

Most upvoted comments

I’ll be updating the Zabbly repository with cherry picked version of the two needed fixes later today.

Right, so the LXC fix above combined with https://github.com/lxc/incus/pull/1895 fixes it here

@mihalicyn https://github.com/lxc/lxc/pull/4536 may do the trick. Going to push that to our daily packages and re-check with both the OCI and nvidia cases.

@mihalicyn I think the dumpable change may have caused this.

I’ve confirmed that reverting to 6.10.1 doesn’t seem to help and we didn’t bump nvidia-container recently. There is a bug in Incus 6.11 that causes part of this issue but even with a fix in place, the hook still fails for me during a setns call.