buildah: ERRO[0000] 'overlay' is not supported over overlayfs

When I execute within a fedora:28 docker container this command

sudo buildah --debug images

I get this error

DEBU[0000] [graphdriver] trying provided driver "overlay" 
DEBU[0000] overlay: override_kernelcheck=true           
ERRO[0000] 'overlay' is not supported over overlayfs    
DEBU[0000] [graphdriver] trying provided driver "overlay" 
DEBU[0000] overlay: override_kernelcheck=true           
ERRO[0000] 'overlay' is not supported over overlayfs    
ERRO[0000] 'overlay' is not supported over overlayfs: backing file system is unsupported for this graph driver
'overlay' is not supported over overlayfs: backing file system is unsupported for this graph driver

Here is the dockerfile used to create the buildah container on Macos

FROM fedora:28

RUN dnf install buildah sudo -y

#
# Create a user and group used to launch processes
# The user ID 1000 is the default for the first "regular" user on Fedora/RHEL,
# so there is a high chance that this ID will be equal to the current user
# making it easier to use volumes (no permission issues)
#
RUN groupadd -r default -g 1000 && useradd -u 1000 -r -g default -m -d /home/me -s /sbin/nologin -c "Default user" default

#
# Add user to sudo
#
RUN echo 'default:secret' | chpasswd
RUN echo '%default ALL=(ALL) ALL' >> /etc/sudoers

# Set the working directory to default' user home directory
WORKDIR /home/me
USER default

Docker info (command executed within macos’s terminal)

docker info
Containers: 67
 Running: 11
 Paused: 0
 Stopped: 56
Images: 160
Server Version: 1.13.1
Storage Driver: overlay2
 Backing Filesystem: xfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: journald
Cgroup Driver: systemd
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: 
Swarm: inactive
Runtimes: docker-runc runc
Default Runtime: docker-runc
Init Binary: docker-init
containerd version:  (expected: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1)
runc version: N/A (expected: 9df8b306d01f59d3a8029be411de015b7304dd8f)
init version: N/A (expected: 949e6facb77383876aeff8a6944dde66b3089574)
Security Options:
 seccomp
  Profile: default
 selinux
Kernel Version: 3.10.0-693.21.1.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.954GiB
Name: minishift
ID: PWZQ:O5OI:AMTH:2IVK:E3FW:IX2G:BJ6X:JTWS:MW2N:YCY3:BDUI:LFAZ
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
 provider=xhyve
Experimental: false
Insecure Registries:
 172.30.0.0/16
 127.0.0.0/8
Live Restore Enabled: false

About this issue

Original URL
State: closed
Created 6 years ago
Comments: 32 (16 by maintainers)

Most upvoted comments

We are seeing issues in podman 3.1 and new Buildah with containers-storage on certain platforms not using fuse-overlayfs and falling back to overlay, which could be this issue.

rhatdan on Apr 6, 2021

docker run --privileged -it -v /tmp/csivvv:/var/lib/containers --rm centos:centos7 bash

you’ll need at least to add the /dev/fuse device to the container (--device /dev/fuse)

giuseppe on Jan 12, 2021

You need to volume mount storage on top of /var/lib/containers for this to work. Or you could use a storage driver of something other then overlay, like vfs.

rhatdan on Jul 17, 2018

Thanks all, this worked for me to run buildah inside a container:

docker run --privileged -it -v /tmp/csivvv:/var/lib/containers --rm  centos:centos7 bash
# # mount |grep containers
/dev/nvme0n1p2 on /var/lib/containers type ext4 (rw,relatime,errors=remount-ro)

Just use /var/lib/containers as a volume.

And I wonder how to use fuse-overlayfs inside of the container to let it works, I have tried but failed:

[root@9237b68aec97 /]# mkdir lower upper work merged
[root@9237b68aec97 /]# 
[root@9237b68aec97 /]# fuse-overlayfs -o lowerdir=lower,upperdir=upper,workdir=work /var/lib/containers
[root@9237b68aec97 /]# mount
overlay on / type overlay (rw,relatime,lowerdir=/var/lib/docker/overlay2/l/YEOKEZEWHFHL7WMM27BXFNQU6T:/var/lib/docker/overlay2/l/C4XFPXQ3OPSOKWEVRJ5B6FT5UW,upperdir=/var/lib/docker/overlay2/20f6a7c025dd2c6bdeebc32c00a561bb49119682c828c271432653f13f90c068/diff,workdir=/var/lib/docker/overlay2/20f6a7c025dd2c6bdeebc32c00a561bb49119682c828c271432653f13f90c068/work,xino=off)
fuse-overlayfs on /var/lib/containers type fuse.fuse-overlayfs (rw,nodev,noatime,user_id=0,group_id=0,default_permissions,allow_other)
[root@9237b68aec97 /]# buildah from --name myvolume --pull reg.docker.alibaba-inc.com/liubin/pouch-volume:latest
ERRO 'overlay' is not supported over <unknown> at "/var/lib/containers/storage/overlay" 
kernel does not support overlay fs: 'overlay' is not supported over <unknown> at "/var/lib/containers/storage/overlay": backing file system is unsupported for this graph driver

Any suggestion? @giuseppe @rhatdan

liubin on Jan 12, 2021

yes, fuse-overlayfs should work on top of overlay.

giuseppe on Nov 25, 2020

Set up fuse-overlayfs inside of the container, I think fuse-overlay will work on overlay. @giuseppe Correct?

rhatdan on Nov 25, 2020