rook: OpenShift: insufficient permission inside the containers

Bug Report

What happened:

When trying to create a cluster the operator fails with:

op-cluster: failed to create cluster in namespace rook. failed to start the mons. failed to initialize ceph cluster info. failed to get cluster info. failed to create mon secret
s. failed to create dir /var/lib/rook/rook. mkdir /var/lib/rook: permission denied

What you expected to happen:

Cluster creation should succeed.

Additional information:

OpenShift uses the following feature to get fewer user privileges on application development where the expected user is ‘root’, see https://blog.openshift.com/jupyter-on-openshift-part-6-running-as-an-assigned-user-id/

How to reproduce it (minimal and precise):

Simply run kubectl create -f rook-cluster.yml

Environment:

OS (e.g. from /etc/os-release): CentOS Linux release 7.4.1708 (Core)
Kernel (e.g. uname -a): Linux k8s-master.example.com 3.10.0-693.11.1.el7.x86_64 #1 SMP Mon Dec 4 23:52:40 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Cloud provider or hardware configuration: VM 1 CPU, 4GB RAM
Rook version (use rook version inside of a Rook Pod): v0.6.0-80.g3dfb151
Kubernetes version (use kubectl version): v1.7.6+a08f5eeb62
Kubernetes cluster type (e.g. Tectonic, GKE, OpenShift): OpenShift
Ceph status (use ceph health in the Rook toolbox): no cluster yet

About this issue

Original URL
State: closed
Created 7 years ago
Comments: 49 (46 by maintainers)

Most upvoted comments

FWIW, I got Rook running on openshift (minishift) by doing the following:

minishift start --openshift-version v3.7.0 --disk-size 100GB
oc login -u system:admin

Now edit the restricted SCC policy as follows:

oc edit scc restricted

change the following values:

allowHostDirVolumePlugin: true
allowHostNetwork: true
allowPrivilegedContainer: true
runAsUser:
  type: RunAsAny
seLinuxContext:
  type: RunAsAny

Now launch the rook operator:

oc create -f rook-operator.yaml

finally modify the dataDirHostPath to point to /mnt/sda1/rook since this is the only path in minishift that actually has space (vs. tmpfs).

and then run

 oc create -f rook-cluster.yaml

the Rook cluster is up:

13:28 $ oc get pods -o wide --all-namespaces
NAMESPACE     NAME                              READY     STATUS      RESTARTS   AGE       IP             NODE
default       docker-registry-1-jsxxp           1/1       Running     0          7m        172.17.0.5     localhost
default       persistent-volume-setup-ph9jd     0/1       Completed   0          7m        172.17.0.2     localhost
default       router-1-cwrv7                    1/1       Running     0          7m        192.168.64.5   localhost
rook-system   rook-agent-4tq87                  1/1       Running     0          4m        192.168.64.5   localhost
rook-system   rook-operator-1614830704-459kb    1/1       Running     0          4m        172.17.0.2     localhost
rook          rook-api-2223072509-frknh         1/1       Running     0          57s       172.17.0.9     localhost
rook          rook-ceph-mgr0-1443125741-rxnth   1/1       Running     0          59s       172.17.0.7     localhost
rook          rook-ceph-mgr1-1307895259-kqxwj   1/1       Running     0          58s       172.17.0.8     localhost
rook          rook-ceph-mon0-xl9k8              1/1       Running     0          1m        172.17.0.3     localhost
rook          rook-ceph-mon1-lz7z9              1/1       Running     0          1m        172.17.0.4     localhost
rook          rook-ceph-mon2-nzsp2              1/1       Running     0          1m        172.17.0.6     localhost
rook          rook-ceph-osd-d9nsl               1/1       Running     0          57s       172.17.0.10    localhost

bassam on Dec 15, 2017