dockerspawner: SwarmSpawner, InvalidArgument: Incompatible options have been provided for the bind type mount.

Trying to deploy hub in Docker Swarm on Docker UCP. We are able to to get the main hub up and running, but when we try to spawn the single user notebooks we get the following:

EAE_jupyterhub.1.zluhmt93kq7u@linuxkit-025000000001    |     Traceback (most recent call last):
EAE_jupyterhub.1.zluhmt93kq7u@linuxkit-025000000001    |       File "/opt/conda/lib/python3.6/site-packages/tornado/web.py", line 1592, in _execute
EAE_jupyterhub.1.zluhmt93kq7u@linuxkit-025000000001    |         result = yield result
EAE_jupyterhub.1.zluhmt93kq7u@linuxkit-025000000001    |       File "/opt/conda/lib/python3.6/site-packages/jupyterhub/handlers/base.py", line 1052, in get
EAE_jupyterhub.1.zluhmt93kq7u@linuxkit-025000000001    |         await self.spawn_single_user(user)
EAE_jupyterhub.1.zluhmt93kq7u@linuxkit-025000000001    |       File "/opt/conda/lib/python3.6/site-packages/jupyterhub/handlers/base.py", line 705, in spawn_single_user
EAE_jupyterhub.1.zluhmt93kq7u@linuxkit-025000000001    |         timedelta(seconds=self.slow_spawn_timeout), finish_spawn_future
EAE_jupyterhub.1.zluhmt93kq7u@linuxkit-025000000001    |       File "/opt/conda/lib/python3.6/site-packages/jupyterhub/handlers/base.py", line 626, in finish_user_spawn
EAE_jupyterhub.1.zluhmt93kq7u@linuxkit-025000000001    |         await spawn_future
EAE_jupyterhub.1.zluhmt93kq7u@linuxkit-025000000001    |       File "/opt/conda/lib/python3.6/site-packages/jupyterhub/user.py", line 489, in spawn
EAE_jupyterhub.1.zluhmt93kq7u@linuxkit-025000000001    |         raise e
EAE_jupyterhub.1.zluhmt93kq7u@linuxkit-025000000001    |       File "/opt/conda/lib/python3.6/site-packages/jupyterhub/user.py", line 409, in spawn
EAE_jupyterhub.1.zluhmt93kq7u@linuxkit-025000000001    |         url = await gen.with_timeout(timedelta(seconds=spawner.start_timeout), f)
EAE_jupyterhub.1.zluhmt93kq7u@linuxkit-025000000001    |       File "/opt/conda/lib/python3.6/site-packages/dockerspawner/dockerspawner.py", line 691, in start
EAE_jupyterhub.1.zluhmt93kq7u@linuxkit-025000000001    |         obj = yield self.create_object()
EAE_jupyterhub.1.zluhmt93kq7u@linuxkit-025000000001    |       File "/opt/conda/lib/python3.6/site-packages/dockerspawner/swarmspawner.py", line 173, in create_object
EAE_jupyterhub.1.zluhmt93kq7u@linuxkit-025000000001    |         mounts=self.mounts,
EAE_jupyterhub.1.zluhmt93kq7u@linuxkit-025000000001    |       File "/opt/conda/lib/python3.6/site-packages/dockerspawner/swarmspawner.py", line 110, in mounts
EAE_jupyterhub.1.zluhmt93kq7u@linuxkit-025000000001    |         for host_loc, vol in self.volume_binds.items()
EAE_jupyterhub.1.zluhmt93kq7u@linuxkit-025000000001    |       File "/opt/conda/lib/python3.6/site-packages/dockerspawner/swarmspawner.py", line 110, in <listcomp>
EAE_jupyterhub.1.zluhmt93kq7u@linuxkit-025000000001    |         for host_loc, vol in self.volume_binds.items()
EAE_jupyterhub.1.zluhmt93kq7u@linuxkit-025000000001    |       File "/opt/conda/lib/python3.6/site-packages/docker/types/services.py", line 237, in __init__
EAE_jupyterhub.1.zluhmt93kq7u@linuxkit-025000000001    |         'Incompatible options have been provided for the bind '
EAE_jupyterhub.1.zluhmt93kq7u@linuxkit-025000000001    |     docker.errors.InvalidArgument: Incompatible options have been provided for the bind type mount.

jupyterhub-config had the following lines to handle volumes

notebook_dir = '/home/jovyan'
c.SwarmSpawner.notebook_dir = notebook_dir
c.SwarmSpawner.volumes = {'/mnt/our_shared_path': notebook_dir, 'mode': 'rw'}
c.SwarmSpawner.volume_driver = None

Tried commenting out the ‘SwarmSpawner.volumes’ line but we still see the same error as above. According to binds in swarmspawner.py (https://github.com/jupyterhub/dockerspawner/blob/master/dockerspawner/swarmspawner.py#L99), if len(self.volume_binds) == 0 it should just return [] and there should be no error. We do not create a mount in docker-compose or manually on the host. Using Jupyterhub version 0.9.4. Haven’t been able to find a way around this issue.

About this issue

Original URL
State: closed
Created 6 years ago
Reactions: 1
Comments: 15 (3 by maintainers)

Commits related to this issue

(PATCH) SwarmSpawner, InvalidArgument Error: SwarmSpawner, InvalidArgument: Incompatible options have been provided for the bind type mount. Scenario: https://github.com/jupyterhub/dockerspawner/i... — committed to cmotadev/dockerspawner by cmotadev 3 years ago

Most upvoted comments

@wlohman-surfsara I tried this by overwriting the mounts in the config file:

from dockerspawner import SwarmSpawner
from docker.types import Mount

class SwarmSpawner2(SwarmSpawner):
    @property
    def mounts(self):
        if len(self.volume_binds):
            driver = self.mount_driver_config
            return [
                Mount(
                    target=vol["bind"],
                    source=host_loc,
                    type="bind",
                    read_only=vol["mode"] == "ro",
                    driver_config=None,
                )
                for host_loc, vol in self.volume_binds.items()
            ]

        else:
            return []

c.JupyterHub.spawner_class = SwarmSpawner2

c.SwarmSpawner2.network_name = os.environ['DOCKER_NETWORK_NAME']
c.SwarmSpawner2.extra_host_config = {'network_mode': os.environ['DOCKER_NETWORK_NAME']}

c.SwarmSpawner2.image = 'registry:5000/jupyterhub_singleuser'

c.SwarmSpawner2.notebook_dir = '/home/jovyan'
c.SwarmSpawner2.volumes = {'/data/home/{username}': '/home/jovyan'}
c.SwarmSpawner2.remove_containers = True

When I comment the line with c.SwarmSpawner2.volumes works (without persistency obviously) but when it is on all I get is

Spawner failed to start [status=0]

No logs in the spawned service also, which is there but with 0 replicas, I tried to escalate it but it fails.

vlizanae on Feb 14, 2019

It works now, it was a permissions issue, but this is definitely a bug, should be labeled as such.

vlizanae on Feb 19, 2019

@zhiyuli ,

We subclassed SwarmSpawner and override the mounts method:

    @property
    def mounts(self):
        if len(self.volume_binds):
            driver = self.mount_driver_config
            return [
                Mount(
                    target=vol["bind"],
                    source=host_loc,
                    type="bind",
                    read_only=False,
                    driver_config=None,
                )
                for host_loc, vol in self.volume_binds.items()
            ]

        else:
            return []

smparekh on Oct 21, 2019