moby: created loop devices do not appear in container even when run privileged

Thank you for sharing all those information! They helped me to overcome the issue. I ended up using mknod to create those partition node files:

LOOPDEV=$(losetup --find --show --partscan ${IMAGE_FILE})

# drop the first line, as this is our LOOPDEV itself, but we only want the child partitions
PARTITIONS=$(lsblk --raw --output "MAJ:MIN" --noheadings ${LOOPDEV} | tail -n +2)
COUNTER=1
for i in $PARTITIONS; do
    MAJ=$(echo $i | cut -d: -f1)
    MIN=$(echo $i | cut -d: -f2)
    if [ ! -e "${LOOPDEV}p${COUNTER}" ]; then mknod ${LOOPDEV}p${COUNTER} b $MAJ $MIN; fi
    COUNTER=$((COUNTER + 1))
done

This works ok if you do docker run -v /dev:/dev --privileged .... Otherwise nothing is creating the new device nodes in the container, as the device nodes are created by one of the hotplug mechanisms on the host (usually). You can create them yourself with mknod and it will work.

(You might also be able to mount devtmpfs in the container, which is one way to get device population. Device hotpluf was not really designed for the container use case alas).

@f0o It looks like loopfs will be best the solution if it makes it into the Linux kernel… I am impatiently looking forward to this since I have the same use case.

More info can be found here: https://fossbytes.com/loopfs-linux-file-system/

Further reading: https://lkml.org/lkml/2020/4/8/506

@Vauteck I think it would be great if it could, but it doesn’t.

I gave up and ran it as -v /dev:/dev --privileged to get it to run. I tried libguestfs, got it to work partially, but ended up taking the short exit. I really would like to get the whole thing to run without it, but higher priorities.

As @justincormack said:

Doing the --privileged thing (or the right capabilties) is certainly the easiest route. It assumes that your Docker host has the right modules though, as effectively you are outsourcing part of your code to the kernel.

I never got lkl to do it either; at this point cannot remember how.

@justincormack was this something ever doable with lkl?

Yeah, I know, there really is no easy answer.

What is your use case exactly by the way?

I am building disk images in a container. I used to do it with packer, but that was a pain: booting a VM, no layers for minor changes, very slow, uploads, full OS install, etc.

Now I can create an image, install all of my prerequisites (like any build env), then I use dd if=/dev/zero of=somefile.img bs=... to make an image file (the directory for the image file is volume-mounted so I can get my artifacts), sgdisk to create GPT and partitions, loopback-mount them, then put on any data I want, from basic data all the way up to a full OS build.

Almost everything works fine, but once you start trying to mount loopback volumes, encrypt them, use lvm, it all gets impossible without --privileged and the devices become very difficult.

It is a pity that containers aren’t built natively for this, because this is a great use case.

And I guess the LDN team is fully integrated now; @justincormack is answering the issues. 😃

@klausenbusk Sound like tl;dr - it has security issues and might be blocked in Ubuntu and is not likely to get in.

@Jaypers I would recommend reading https://lkml.org/lkml/2020/4/8/647.

Hi Guys, I am getting a similar problem.

1. Use SUSE docker image.
2. mounted /dev:/dev
3. run container as --privileged
4. execute DEVICE= losetup -f; echo $DEVICE shows a /dev/loop0;
5. execute losetup $DEVICE $DISK,
6. execute losetup -a the last step should have listed /dev/loop0, but doesn’t show any devices. Please help. Thanks.