origin: oc cluster with oc 1.5.0.alpha.2 fails when persistent data directory used.

I’m using:

VirtualBox 5.1.26 Kubernetes v1.5.2+43a9be4
openshift v1.5.0+031cbe4

Didn’t work for me using --host-data-dir (and others) :

oc cluster up  --logging=true --metrics=true --docker-machine=openshift --use-existing-config=true --host-data-dir=/vm/data --host-config-dir=/vm/config --host-pv-dir=/vm/pv --host-volumes-dir=/vm/volumes

With output:

-- Checking OpenShift client ... OK
-- Checking Docker client ...
   Starting Docker machine 'openshift'
   Started Docker machine 'openshift'
-- Checking Docker version ...
   WARNING: Cannot verify Docker version
-- Checking for existing OpenShift container ... OK
-- Checking for openshift/origin:v1.5.0 image ... OK
-- Checking Docker daemon configuration ... OK
-- Checking for available ports ... OK
-- Checking type of volume mount ...
   Using Docker shared volumes for OpenShift volumes
-- Creating host directories ... OK
-- Finding server IP ...
   Using docker-machine IP 192.168.99.100 as the host IP
   Using 192.168.99.100 as the server IP
-- Starting OpenShift container ...
   Starting OpenShift using container 'origin'
FAIL
   Error: could not start OpenShift container "origin"
   Details:
     Last 10 lines of "origin" container log:
     github.com/openshift/origin/vendor/github.com/coreos/pkg/capnslog.(*PackageLogger).Panicf(0xc4202a1600, 0x42b94c0, 0x1f, 0xc4214d9f08, 0x2, 0x2)
        /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/coreos/pkg/capnslog/pkg_logger.go:75 +0x16a
     github.com/openshift/origin/vendor/github.com/coreos/etcd/mvcc/backend.newBackend(0xc4209f84c0, 0x33, 0x5f5e100, 0x2710, 0xc4214d9fa8)
        /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/coreos/etcd/mvcc/backend/backend.go:106 +0x341
     github.com/openshift/origin/vendor/github.com/coreos/etcd/mvcc/backend.NewDefaultBackend(0xc4209f84c0, 0x33, 0x461e51, 0xc421471200)
        /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/coreos/etcd/mvcc/backend/backend.go:100 +0x4d
     github.com/openshift/origin/vendor/github.com/coreos/etcd/etcdserver.NewServer.func1(0xc4204bf640, 0xc4209f84c0, 0x33, 0xc421079a40)
        /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/coreos/etcd/etcdserver/server.go:272 +0x39
     created by github.com/openshift/origin/vendor/github.com/coreos/etcd/etcdserver.NewServer
        /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/coreos/etcd/etcdserver/server.go:274 +0x345

Openshift writes to the directories /vm/… (also defined in VirtualBox) but successfully won’t start.

@bparees or @GrahamDumpleton : Just wondering if you would have any ideas as I’ve seen a couple of your conversations that touched on this problem please?

Thanks.

The first is how do I specify a directory in the VM?

You need to specify a directory that is not a windows directory (which is pretty much anything that’s not /c/blah). In non-windows Docker, it’s easy to just mount the root fs of the vm into a container and explore. However, on Windows, it looks like mounting the root doesn’t work, so using /var is a safe bet. How can you explore it? Simply run a container that mounts it and take a look at what’s there:

docker run -ti --entrypoint=/bin/bash -v /var:/var openshift/origin:latest
$ ls -al /var

Is it enough to just specify a directory outside of that and it lands on the VM?

Pretty much yes, as long as it’s not root (‘/’) as I mentioned above. I tried -v /foo:/foo and it simply creates that directory on the vm

where on the VM would be an appropriate place to store the PV directories?

I’d follow the pattern we already use by default, something under ‘/var/lib/’, like: ‘/var/lib/profiles/[profile]/pv’

And finally, how can I delete the PV directories on the VM when I am destroying a profile?

You can just run a container that mounts the parent directory and remove that directory. For example:

docker run --entrypoint=/bin/bash --rm -v /var:/var openshift/origin:latest -c "rm -rf /var/lib/profiles/myprofile"

Same for copying it to the host drive… just create a container with a given name that mounts the directory, use docker cp to copy the contents, and then delete the container:

docker create --name=mycontainer -v /var:/var openshift/origin:latest
docker cp mycontainer:/var/lib/profiles/myprofile/pv c:\users\graham\profiles\myprofile
docker rm mycontainer