toolbox: Containers remain running after exiting

In particular, this means it’s impossible to remove a toolbox-created container without first stopping/killing it with podman:

 ~/toolbox  ./toolbox create -c test
Created container: test
Enter with: toolbox enter --container test
 ~/toolbox  ./toolbox enter -c test 
🔹[exalm@toolbox toolbox]$ logout
 ~/toolbox  ./toolbox rm test
toolbox: failed to remove container test
 ~/toolbox  podman ps
CONTAINER ID  IMAGE                              COMMAND     CREATED         STATUS             PORTS  NAMES
1a08f09ea797  localhost/fedora-toolbox-exalm:30  sleep +Inf  16 seconds ago  Up 10 seconds ago         test
 ~/toolbox  podman stop test
1a08f09ea79710801859bea8dc6a5a85d2031ce1a73dd7d284c3e1fa51a67be0
 ~/toolbox  ./toolbox rm test
 ~/toolbox  

About this issue

  • Original URL
  • State: open
  • Created 5 years ago
  • Reactions: 9
  • Comments: 21 (10 by maintainers)

Commits related to this issue

Most upvoted comments

Using toolbox rm --force is not able to delete the container when it is running on my machine.

Perhaps what would could be done is to detect if the container is running when deleting it and then give the user a prompt such as This toolbox is currently running, are you sure you wish to delete it [y/N]: Then call podman stop before calling the delete command if they choose to continue.

So far, I can think of two different ways to make Toolbox containers reference-counted so that they automatically stop once the last toolbox enter or toolbox run session has terminated.

Another option, used by coreos/toolbox, is to call podman stop after every invocation of podman exec in toolbox enter. podman stop will keep failing as long there’s any active podman exec session, but once the last one finishes, the container will get stopped.

It’s less sophisticated than the other alternatives, but simpler to implement.

So far, I can think of two different ways to make Toolbox containers reference-counted so that they automatically stop once the last toolbox enter or toolbox run session has terminated.

(Note that stopping the container is the same thing as terminating the container’s entry point process.)

  • The enter and run commands use POSIX signals to tell the container’s entry point that a new session is about to start, or has just ended. eg., it could send SIGUSR1 for one and SIGUSR2 for the other. The entry point handles these signals and keeps a reference count of the number of active sessions. Once the counter hits zero, it terminates.

This can be implemented with Go channels, os/signal and such. Here is an example.

The downside of this is that it’s not resilient against crashes in the enter and run commands. If they crash, then the second signal indicating the end of the session might not get sent.

  • The enter and run sessions acquire shared file locks (ie., flock --shared ...) and the entry point blocks trying to acquire an exclusive lock (ie., flock --exclusive ...) on a common file. The entry point will be unblocked once all shared locks have been released by the active sessions, and then it can terminate.

The nice thing about this is that locks are automatically released by the kernel when a process terminates. So, even if the enter and run commands crash, the locks would get released.