microk8s: Can't connect to Docker daemon when SELinux is in use

Nice talking to you face to face @GregoireW .

Indeed the file system with the binaries is read only and I read the labels are filesystem metadata.

I want to see if I can set labels during the build of the snap package. I will have news on this soon.

Hi @akaihola ,

Thank you for reporting this.

Here is what happens. When dockerd starts it listens on a socket (unix:///var/snap/microk8s/current/docker.sock), selinux is blocking socket communication by default as discussed in https://forum.snapcraft.io/t/selinux-blocking-socket-activation-on-fedora/6931 . You can verify this by temporarily disabling selinux and restarting microk8s:

sudo setenforce permissive
sudo snap stop microk8s
sudo snap start microk8s

I am taking a look now how/if we can detect selinux and set it to allow the required socket communication.

Thank you for taking the time to report this issue and apologies for any inconvenience.

Has this changed at all?

As MicroK8s is a snap most of the AppArmor and SELinux configuration is taken care by the snapcraft underlying layer. On the MicroK8s side we are working towards a strictly confined snap that would comply with the snapcraft best practices and will take advantage of all the security enhancements offered.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@ktsakalozos It was great to meet you at the cfgmgmt camp.

On this subject, I was about to extract all rules I added on my computer, but I know it is not a good option as every container we will create will need a new set of rules.

I decide to check how docker / containerd was doing on the subject. They have a pre-defined set of rules inside an rpm (container-selinux) Those are defined in a repo: https://github.com/containers/container-selinux.

My idea was first to try to set the selinux container type on the docker in the snap, but unfortunatly I got a small issue: Snap are read only, so the chcon -t container_runtime_t /snap/microk8s/current/usr/bin/dockerd return a read only fs error.

Not sure how to test that. It would be cool if installing the snap requires the container-selinux RPM then change the type of docker binaries to the container_runtime_t. It would be cooler if it solves every issue 😉

I ran this command:

audit2allow -M microk8s

and pasted in the lines from audit.log (see my long comment above). It created for me the files microk8s.pp and microk8s.te. The contents of microk8s.te are:

module mypol 1.0;

require {
	type insmod_exec_t;
	type var_run_t;
	type snappy_var_t;
	type proc_net_t;
	type modules_object_t;
	type fs_t;
	type cgroup_t;
	type unlabeled_t;
	type http_cache_port_t;
	type snappy_t;
	type sysctl_net_t;
	type unreserved_port_t;
	class file { create execute execute_no_trans getattr ioctl map open read write };
	class sock_file { getattr setattr unlink write };
	class dir { add_name create getattr mounton write };
	class capability sys_resource;
	class unix_stream_socket connectto;
	class process setsched;
	class tcp_socket { name_bind name_connect };
	class filesystem getattr;
}

#============= snappy_t ==============
allow snappy_t cgroup_t:dir { add_name create write };
allow snappy_t cgroup_t:file { create write };
allow snappy_t fs_t:filesystem getattr;
allow snappy_t http_cache_port_t:tcp_socket name_bind;
allow snappy_t insmod_exec_t:file { execute execute_no_trans getattr open read };
allow snappy_t modules_object_t:file { getattr open read };
allow snappy_t proc_net_t:file { getattr open read };
allow snappy_t self:capability sys_resource;
allow snappy_t self:process setsched;

#!!!! This avc can be allowed using the boolean 'daemons_enable_cluster_mode'
allow snappy_t self:unix_stream_socket connectto;
allow snappy_t snappy_var_t:dir mounton;

#!!!! This avc can be allowed using the boolean 'domain_can_mmap_files'
allow snappy_t snappy_var_t:file map;
allow snappy_t snappy_var_t:sock_file { getattr setattr unlink write };
allow snappy_t sysctl_net_t:dir getattr;
allow snappy_t sysctl_net_t:file { getattr write };

#!!!! This avc can be allowed using the boolean 'domain_can_mmap_files'
allow snappy_t unlabeled_t:file map;
allow snappy_t unlabeled_t:file { execute_no_trans ioctl };
allow snappy_t unreserved_port_t:tcp_socket name_connect;
allow snappy_t var_run_t:file { getattr open read };

If I run

sudo semodule -i microk8s.pp

then microk8s works as described in the README.

I hope this is helpful for enabling support for SELinux enabled systems.