compose: docker-compose run fails for running network_mode=host services

The problem does not go away by doing nothing, then flagging it as stale, and finally closing the ticket. Bugs need to be fixed, not managed

This issue has been automatically marked as not stale anymore due to the recent activity.

Same issue.

This is still relevant 👍

likewise

Same issue

Docker compose v2 doesn’t suffer this limitation

$docker compose version
Docker Compose version v2.4.1
$ docker compose run --rm redis hostname
docker-desktop

Encountered here too

I close the issue, Compose v2 works as expected and Compose v1 is end-of-life Tested with @hholst80 sample

> bat compose.yaml
───────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: compose.yaml
       │ Size: 152 B
───────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ services:
   2   │   foo:
   3   │     image: debian
   4   │     command: sleep inf
   5   │     network_mode: host
   6   │   bar:
   7   │     image: debian
   8   │     command: sleep inf
   9   │     network_mode: host
───────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
> docker compose up -d
[+] Running 3/3
 ⠿ foo Pulled                                                                                                                                                                                                                            5.6s
   ⠿ cfc947b533a3 Pull complete                                                                                                                                                                                                          3.2s
 ⠿ bar Pulled                                                                                                                                                                                                                            5.6s
[+] Running 2/2
 ⠿ Container issue-4548-foo-1  Started                                                                                                                                                                                                   0.4s
 ⠿ Container issue-4548-bar-1  Started

This is also happening in https://github.com/akvo/akvo-rsr where network_mode: service:main_network is being used.

$ docker-compose up -d
Creating akvo-rsr_rsr-memcached_1 ... done
Creating akvo-rsr_rsrdbhost_1     ... done
Creating akvo-rsr_mainnetwork_1   ... done
Creating akvo-rsr_web_1           ... done
Creating akvo-rsr_reports_1       ... done
Creating akvo-rsr_nginx_1         ... done
$ docker-compose run --rm web echo lol
Starting akvo-rsr_mainnetwork_1 ... done
ERROR: Cannot create container for service web: conflicting options: container type network can't be used with links. This would result in undefined behavior

I’m also running into the exact same problem. Here are my versions:

docker-compose version 1.11.2, build dfed245
Docker version 17.03.1-ce, build c6d412e

and here’s the gist of my docker-compose.yml file:

(some part redacted)

version: '3'

services:

  backend:
    container_name: backend
    build:
      context: .
      dockerfile: DockerfileBackend
    image: 123456.ecr.some-region.amazonaws.com/some-repo:latest
    environment:
      - DATABASE_URL=postgres://postgres:P@ssw0rd@localhost:5432/some_db
    volumes:
      - ./frontend_assets/static_files:/code/static_files
    command: gunicorn --bind 0.0.0.0:8000 --workers 3 --worker-class gevent app.config.wsgi:application --log-level=INFO
    ports:
      - "8000:8000"
    network_mode: "host"

  frontend:
    container_name: frontend
    build:
      context: .
      dockerfile: DockerfileFrontend
    image: 123456.dkr.ecr.some-region.amazonaws.com/frontend-repo:latest
    volumes:
      - ./frontend_assets:/code/frontend_assets

With this docker-compose.yml, the error will happen when the backend container is already running and we attempt to execute another docker run command related to backend.

Here’s a concrete example. Assuming that backend is already running, this is the behavior:

# When running `frontend` container, there's no problem
docker-compose run frontend cp -rf /code/static_files/ /code/frontend_assets/
# But once you run the `backend` container while there's already one running...
docker-compose run backend python manage.py collectstatic --noinput

You will immediately see this error:

ERROR: Cannot create container for service backend: Conflicting options: host type networking can't be used with links. This would result in undefined behavior

---

Workaround

There’s a workaround for this issue that works for me, which is to do a docker-compose down prior to doing docker-compose run backend. Eg, this is what I do now:

docker-compose down
docker-compose run frontend cp -rf /code/static_files/ /code/frontend_assets/
# No problem. We are happy! ^_^
docker-compose run backend python manage.py collectstatic --noinput

Same problem here … unbelievable how after years this is still present… using docker-compose up

I am also seeing the same issue in a different use-case, is there any way to disable the links docker-compose creates automatically? What is the way forward on this issue? It is open since 3 years, is there any way we can escalate this?

I got the same issue. Trying to run a one-off command to service previously started via docker-compose with network_mode: host fails with error.

# docker-compose --version
docker-compose version 1.25.0, build 0a186604
#  docker-compose run webserver ps
ERROR: Cannot create container for service webserver: conflicting options: host type networking can't be used with links. This would result in undefined behavior

verbose output:

compose.config.config.find: Using configuration files: /home/project/frontend-docker-compose-start-config.latest
docker.utils.config.find_config_file: Trying paths: ['/home/.docker/config.json', '/home/.dockercfg']
docker.utils.config.find_config_file: Found file at path: /home/.docker/config.json
docker.auth.load_config: Found 'auths' section
docker.auth.parse_auth: Found entry (registry='****', username='AWS')
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.38/version HTTP/1.1" 200 567
compose.cli.command.get_client: docker-compose version 1.25.0, build 0a186604
docker-py version: 4.1.0
CPython version: 3.7.4
OpenSSL version: OpenSSL 1.1.0l  10 Sep 2019
compose.cli.command.get_client: Docker base_url: http+docker://localhost
compose.cli.command.get_client: Docker version: Platform={'Name': ''}, Components=[{'Name': 'Engine', 'Version': '18.09.9-ce', 'Details': {'ApiVersion': '1.39', 'Arch': 'amd64', 'BuildTime': '2019-11-01T19:28:24.000000000+00:00', 'Experimental': 'false', 'GitCommit': '039a7df', 'GoVersion': 'go1.10.3', 'KernelVersion': '4.14.146-120.181.amzn2.x86_64', 'MinAPIVersion': '1.12', 'Os': 'linux'}}], Version=18.09.9-ce, ApiVersion=1.39, MinAPIVersion=1.12, GitCommit=039a7df, GoVersion=go1.10.3, Os=linux, Arch=amd64, KernelVersion=4.14.146-120.181.amzn2.x86_64, BuildTime=2019-11-01T19:28:24.000000000+00:00
compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('project_default')
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.38/networks/project_default HTTP/1.1" 404 55
compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('project_default')
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.38/networks/project_default HTTP/1.1" 200 822
compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {'Attachable': True,
 'ConfigFrom': {'Network': ''},
 'ConfigOnly': False,
 'Containers': {'5079456ae0bf67c3eddfec0dc888bb6926a1d7c97a18515260582f765297cc17': {'EndpointID': 'b89c45692b6a4ab4106111185474b03363e0bae22ddbe95689e0d91da225fac5',
                                                                                     'IPv4Address': '192.168.240.2/20',
                                                                                     'IPv6Address': '',
                                                                                     'MacAddress': '02:42:c0:a8:f0:02',
                                                                                     'Name': 'config'}},
 'Created': '2020-04-07T15:56:31.879957006Z',
 'Driver': 'bridge',
...
compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('****:master')
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.38/images/****:master/json HTTP/1.1" 200 None
compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64',
 'Author': '',
 'Comment': 'buildkit.dockerfile.v0',
 'Config': {'ArgsEscaped': True,
            'AttachStderr': False,
            'AttachStdin': False,
            'AttachStdout': False,
            'Cmd': ['/bin/sh',
                    '-c',
                    '/bin/sh -c "envsubst \'${SERVER_NAME}\' < '
...
compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=project', 'com.docker.compose.service=webserver', 'com.docker.compose.oneoff=False']})
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.38/containers/json?limit=-1&all=0&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dproject%22%2C+%22com.docker.compose.service%3Dwebserver%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D HTTP/1.1" 200 1596
compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 1 items)
compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('db2fcd900b77b77342a44535fbc063d93609bb43467b21e3c37678fe9db235f5')
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.38/containers/db2fcd900b77b77342a44535fbc063d93609bb43467b21e3c37678fe9db235f5/json HTTP/1.1" 200 None
compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '',
 'Args': ['-c',
          '/bin/sh -c "envsubst \'${SERVER_NAME}\' < '
          '/etc/nginx/conf.d/*** > /etc/nginx/conf.d/server.conf '
          '&& exec nginx -g \'daemon off;\'"'],
 'Config': {'ArgsEscaped': True,
            'AttachStderr': False,
            'AttachStdin': False,
            'AttachStdout': False,
            'Cmd': ['/bin/sh',
...
compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[('webserver', 'webserver')], port_bindings={}, binds=['/etc/ssl/certs/****:/etc/ssl/certs/****:rw'], volumes_from=[], privileged=False, network_mode='host', devices=None, dns=None, dns_opt=None, dns_search=None, restart_policy=None, runtime=None, cap_add=None, cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': 'json-file', 'Config': {'max-file': '10', 'max-size': '200k'}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=None, ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=None, device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None)
compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': ['/etc/ssl/certs/****:/etc/ssl/certs/****:rw'],
 'Links': ['webserver:webserver'],
 'LogConfig': {'Config': {'max-file': '10', 'max-size': '200k'},
               'Type': 'json-file'},
 'NetworkMode': 'host',
 'PortBindings': {},
 'VolumesFrom': []}
compose.cli.verbose_proxy.proxy_callable: docker create_container <- (environment=[], image='****', volumes={'/etc/ssl/certs/***': {}}, command=['ps'], tty=True, stdin_open=True, detach=False, ports=[], name='project_webserver_run_1ce47b915946', labels={'com.docker.compose.project': 'project', 'com.docker.compose.service': 'webserver', 'com.docker.compose.oneoff': 'True', 'com.docker.compose.project.working_dir': '/home/project', 'com.docker.compose.project.config_files': '/home/project/frontend-docker-compose-start-config.latest', 'com.docker.compose.slug': '1ce47b915946136712ea2c7d0731230e1223b6b90cd273a6509286fb1f58936', 'com.docker.compose.version': '1.25.0'}, host_config={'NetworkMode': 'host', 'VolumesFrom': [], 'Binds': ['/etc/ssl/certs/***:/etc/ssl/certs/***:rw'], 'PortBindings': {}, 'Links': ['webserver:webserver'], 'LogConfig': {'Type': 'json-file', 'Config': {'max-file': '10', 'max-size': '200k'}}})
urllib3.connectionpool._make_request: http://localhost:None "POST /v1.38/containers/create?name=project_webserver_run_1ce47b915946 HTTP/1.1" 400 122

Otherwise speaking, compose is trying to create a temporary container for a one-off command and link it to the existing container. Proper fix could be passing --network-mode host to a run command and don’t use the link when it is passed.

Workaround: don’t mess with the run, use exec:

PID   USER     TIME  COMMAND
    1 root      0:00 nginx: master process nginx -g daemon off;
    7 nginx     0:33 nginx: worker process
    8 nginx     0:00 nginx: worker process
   14 root      0:00 sh
   39 root      0:00 ps

I am also encountering the same problem on a container with network_mode: host.

As a workaround, I had to stop my running containers, then run my separate run command.

$ docker-compose up -d
docker-compose_bar_1 is up-to-date
docker-compose_foo_1 is up-to-date
$ docker-compose run --rm bar echo hello world
ERROR: Cannot create container for service bar: conflicting options: host type networking can't be used with links. This would result in undefined behavior
$ docker-compose stop
$ docker-compose run --rm bar echo hello world
hello world
$

Two and a half years later, we still have the same problem.

$ docker-compose version
docker-compose version 1.24.1, build 4667896b
docker-py version: 3.7.3
CPython version: 3.6.8
OpenSSL version: OpenSSL 1.1.0j  20 Nov 2018
$ cat docker-compose.yml
version: '2'

services:
  foo:
    image: debian
    command: sleep inf
    network_mode: host
  bar:
    image: debian
    command: sleep inf
    network_mode: host
$ docker-compose up -d
docker-compose_bar_1 is up-to-date
docker-compose_foo_1 is up-to-date
$ docker-compose run --rm bar echo hello world
ERROR: Cannot create container for service bar: b"conflicting options: host type networking can't be used with links. This would result in undefined behavior"
$ 

@nathantsoi docker-compose run ... does not enter the container, it makes a new container with the same settings. To “enter” a running container you would use docker-compose exec.

The reason turned out to be a misleading error message in my case. An instance of the container was in restart mode, and docker compose got confused (I have not links, but network_mode=host)