moby: docker-compose not working with swarm overlay networking in docker v1.12rc
Output of docker version
:
root@xxxxx:~# docker version
Client:
Version: 1.12.0-rc2
API version: 1.24
Go version: go1.6.2
Git commit: 906eacd
Built: Fri Jun 17 21:09:07 2016
OS/Arch: linux/amd64
Server: Version: 1.12.0-rc2 API version: 1.24 Go version: go1.6.2 Git commit: 906eacd Built: Fri Jun 17 21:09:07 2016 OS/Arch: linux/amd64
root@xxxxx:~# docker-compose version docker-compose version 1.8.0-rc1, build 9bf6bc6 docker-py version: 1.8.1 CPython version: 2.7.9 OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
Output of docker info
:
Containers: 49
Running: 0
Paused: 0
Stopped: 49
Images: 109
Server Version: 1.12.0-rc2
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 246
Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge overlay null host
Swarm: active
NodeID: 34lgngb2j2iufvk046th0p6qq
IsManager: Yes
Managers: 1
Nodes: 3
CACertHash: sha256:4dd1a761493f5ef3fcaa783113b85b935a7686cc954f5507af9b3743b87cd5b9
Runtimes: default
Default Runtime: default
Security Options: apparmor seccomp
Kernel Version: 3.16.0-30-generic
Operating System: Ubuntu 14.04.2 LTS
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.955 GiB
Name: pek2-gosv-16-dhcp226
ID: 2B4M:L4M2:6Q3A:HLAU:3CNJ:JA7I:34T3:NRBA:BGY6:RY5A:4AS3:AC6L
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Http Proxy: xxxx.some.com:1234
Https Proxy: xxxx.some.com:1234
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Insecure Registries:
127.0.0.0/8
Additional environment details (AWS, VirtualBox, physical, etc.):
Steps to reproduce the issue: 0. Prepare three ubuntu14.04 machines, here I used VMware VMs
- Upgrade docker engine to v1.12rc2
- (Optional)Upgrade docker compose 1.8.0-rc1 (I could also reproduce this issue on earlier version such as v1.6.1GA)
- create a swarm cluster with 1 manger node and 2 slave nodes, as documented in v1.12 swarm-mode doc. Here in my network setup, “ingress” is swarm created overlay network pengzhencao@pek2-gosv-16-dhcp226:~/docker/docker$ docker node list ID NAME MEMBERSHIP STATUS AVAILABILITY MANAGER STATUS 34lgngb2j2iufvk046th0p6qq * pek2-gosv-16-dhcp226 Accepted Ready Active Leader 37fmlm52xw0iy7k6zoo5j09px pek2-gosv-16-dhcp160 Accepted Ready Active 61lw7iskl9ivnxaveby8dfaf1 pek2-gosv-16-dhcp45 Accepted Ready Active pengzhencao@pek2-gosv-16-dhcp226:~/docker/docker$ docker network list NETWORK ID NAME DRIVER SCOPE 8b94d08c0f67 bridge bridge local 74b71c054864 deploy_default bridge local abd08b2c6c5d docker_gwbridge bridge local 651d9df026eb host host local 7oywymoz6bbp ingress overlay swarm 18e3039be16b none null local
- On the swarm manager node, edit docker-compose.yml file with the following settings: version: ‘2’ services: log: build: ./log/ volumes: - /var/log/harbor/:/var/log/docker/ ports: - 1514:514 …(some other services) networks: default: external: name: ingress
- run with “docker-compose up”
Describe the results you received:
pengzhencao@pek2-gosv-16-dhcp226:~/harbor/Deploy$ docker-compose up
Recreating deploy_log_1
ERROR: for log swarm-scoped network (ingress) is not compatible with docker create
or docker run
. This network can be only used docker service
ERROR: Encountered errors while bringing up the project.
Describe the results you expected:
Docker compose should deploy the services up with overlay network “ingress”
Additional information you deem important (e.g. issue happens only occasionally): I’ve followed swarm and docker-compose integration guide here: https://docs.docker.com/compose/networking/
The above error seems to be thrown in L356:
in docker/daemon/container_operations.go
330 func errClusterNetworkOnRun(n string) error {
331 return fmt.Errorf(“swarm-scoped network (%s) is not compatible with docker create
or docker run
. This network can be only used docker service”, n)
332 }
333
334 // updateContainerNetworkSettings update the network settings
335 func (daemon _Daemon) updateContainerNetworkSettings(container *container.Container, endpointsConfig map[string]_networktypes.EndpointSetti ngs) error {
336 var (
337 n libnetwork.Network
338 err error
339 )
340
341 mode := container.HostConfig.NetworkMode
342 if container.Config.NetworkDisabled || mode.IsContainer() {
343 return nil
344 }
345
346 networkName := mode.NetworkName()
347 if mode.IsDefault() {
348 networkName = daemon.netController.Config().Daemon.DefaultNetwork
349 }
350 if mode.IsUserDefined() {
351 n, err = daemon.FindNetwork(networkName)
352 if err != nil {
353 return err
354 }
355 if !container.Managed && n.Info().Dynamic() {
356 return errClusterNetworkOnRun(networkName)
357 }
358 networkName = n.Name()
359 }
360
About this issue
- Original URL
- State: closed
- Created 8 years ago
- Comments: 23 (11 by maintainers)
My understanding is that docker-compose is getting replaced by .dab files for the multi-node use case, which is fine… The problem is that, at the moment, a lot of features (especially security features like capabilities, etc. but many others https://github.com/docker/compose/issues/3680#issuecomment-240143521 ) are not there yet, even from the docker service CLI. It’s quite visible when you run docker-compose bundle.
So beside scaling a simple service with almost no configuration, I’m curious how people are actually using swarm in production, securely, right now? And how do you do it without typing commands by hand?
Maybe our use case is too complex, though it doesn’t seem like it, but hopefully every arguments supported by run that were available in compose will be in the .dab by 1.13, otherwise it’s going to be a long wait… Cheers!
@pouicr we’re having similar issues. So far, from what I can find, it appears that you can only connect to a Docker 1.12 ‘swarm scoped’ overlay network if you use
docker service
(see https://github.com/docker/docker/issues/23901#issuecomment-228133810). I presume this will work indocker bundle
as well, but we’re not ready to shift to that yet in Production given its experimental status.Happy to be corrected. We are trying to shift to using a Docker 1.12 swarm, but our overlay network in the
docker-compose.yml
file is throwing the same issue experienced by @michelvocks:For our current concept testing we have had to use a
bridge
network driver scoped tolocal
. This was done by commenting out the driver in ourdocker-compose.yml
file:@schmunk42 I think it’s currently client-side only (https://github.com/docker/docker/blob/e8b9f38c7e2e249084c78c81f136c6d22432af6f/cli/command/stack/deploy.go#L48-L69)
Thanks @sdemura for your input. And @dnephin “docker-compose bundle” does not work either with same error.
hmm. Will docker-compose be supported with built-in swarm-mode overlay network when v1.12 is GA? Otherwise it will still be complex to deploy another external K-V store. If it not possible to make it in v1.12GA, we need to make it clearly in docker-compose doc that user should use traditional swarm cluster rather than built-in swarm-mode for this overlay network scenario.