compose: [BUG] incorrect label com.docker.compose.network

Description

I see that a change for better diagnostic message on network label mismatch by @ndeloof in https://github.com/docker/compose/pull/10639 is possibly creating the following errors when I’m updating zabbix or gitlab using version 2.19.1. I am able to update when using docker-compose version 2.18.1

network zabbix was found but has incorrect label com.docker.compose.network set to "zabbix"

        "Labels": {
            "com.docker.compose.network": "zabbix",
            "com.docker.compose.project": "zabbix-server",
            "com.docker.compose.version": "1.25.5"
        }

network gitlab was found but has incorrect label com.docker.compose.network set to "gitlab"

        "Labels": {
            "com.docker.compose.network": "gitlab",
            "com.docker.compose.project": "gitlab",
            "com.docker.compose.version": "1.25.5"
        }

I’m not sure how to fix this. Could the error be a bit more descriptive? Can I delete the com.docker.compose.network label or must it be another value?

Steps To Reproduce

docker-compose --env-file=/opt/docker/.env.prod.aws -f /opt/devops/zabbix-server/docker-compose.yml up -d --force-recreate

Compose Version

2.19.1

Docker Environment

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
  scan: Docker Scan (Docker Inc.)

Server:
 Containers: 5
  Running: 5
  Paused: 0
  Stopped: 0
 Images: 17
 Server Version: 20.10.6
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Native Overlay Diff: false
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: d71fcd7d8303cbf684402823e425e9dd2e99285d
 runc version: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.4.17-2136.320.7.1.el7uek.x86_64
 Operating System: CentOS Linux 7 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 15.12GiB
 Name: gaewp-dkl
 ID: NDCY:5WEL:3U23:VFFD:QE7A:ZFZW:3RW3:Q5O3:7NLE:5LQ4:JXDF:5YR4
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

No response

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Reactions: 15
  • Comments: 25

Commits related to this issue

Most upvoted comments

I had the same “incorrect label” issue and after a docker system prune, everything went back to normal.

TL;DR: Not sure this will solve every issue regarding this, but I was able to remove my existing network and recreate it with the default label to get past this issue.

docker network rm networkname
docker network create networkname --label "com.docker.compose.network=default"

I ran into this as well. It looks like the change causing this was actually https://github.com/docker/compose/pull/10612, also by @ndeloof - the one mentioned in the OP is regarding a warning that’s similar but different, but this change is what adds the error itself.

The error I was seeing is the same as @rtaft’s comment.

I don’t have any labels set, so if inspect.Labels[api.NetworkLabel] is "" then I thought my expectedNetworkLabel would also be "". The error message is not showing the expectedNetworkLabel, so I wasn’t sure what it was.

hopefully-relevant excerpt from compose file:

services:
  ...

  proxy:
    ...
    networks:
      - a_default
      - b_default

networks:
  a_default:
    external: true
  b_default:
    external: true

If I docker network rm both networks and try again, I get network a_default declared as external, but could not be found, as I would expect. I then tried to recreate them with docker network create and guessed at a label name, ex: --label "com.docker.compose.network=a_default", but I got network a_default was found but has incorrect label com.docker.compose.network set to "a_default"

Then, I tried with default as the network value: docker network create a_default --label "com.docker.compose.network=default" and it seemed to work. So, that’s my guess as to what expectedNetworkLabel is.

I did rm this network and create a new one to see if I could consistently reproduce it, and it did again get past this error, but then I got an error: my app was still referencing that first removed network instead of the new one. So, I did a docker system prune and was able to reproduce by creating a new network with the default label. My b_default network was able to still be created without a label.

I looked at what I believe to be the networking documentation (1/2/3/4) but I can’t find where it says that at least one default label is now required when creating networks. (Admittedly, I could be looking in the wrong place or for the wrong thing.)

Is this now necessary, or could the code handle for this case without having to set a default label when creating the network? If it is necessary, I think it could be added (or added more clearly) to the documentation.

I also think I am correctly setting the networks as external: true, though I’m also seeing that warning, but since it’s just a warning, it’s not blocking me.

I tried to be thorough in looking into this, since I wasn’t very familiar with this topic – apologies if I got anything wrong and thanks in advance for any help!

I had this same issue after updating to the latest docker/compose versions on Ubuntu 22.04. I ended up pruning all networks with all my services down and after bringing them up again, voila! Networks were created successfully and services started without issues

Same issue. Downgrading to 2.18.1 solved it.

If you are going to do the downgrade as a workaround, don’t expect (like me 😅) that downgrading docker will downgrade the compose plugin, at least in Ubuntu using the docker-ce distribution you have to downgrade the compose plugin like this:

sudo apt install docker-compose-plugin=2.18.1-1~ubuntu.20.04~focal

My error: network net1 was found but has incorrect label com.docker.compose.network set to "net1".

This problem is present on v2.20.0 and v2.19.1. It is not present on v2.18.1.

My docker-compose file’s networks looks like this:

networks:
  default:
    name: "net1"

  net1:
    name: "net1"
    ipam:
      config:
        - subnet: "172.21.0.0/16"

  internalnet:
    name: "internalnet"
    driver: "bridge"
    internal: true

  vpn:
    name: "vpn"
    driver: "bridge"
    ipam:
      config:
        - subnet: "172.10.0.0/16"

These networks are then used by the services based on connectivity requirements.

I used

networks:
  default:
    name: "net1"

To set the default network net1 for all containers. With the introduction of v2.19.1, this stopped working.

For those in need for a solution on this one, here are my findings:

First, I had to do a docker network ls --format=json in order to understand the problem:

{"CreatedAt":"2023-09-06 14:29:15.821708941 +0200 CEST",
"Driver":"bridge",
"ID":"0a470b25babd",
"IPv6":"false",
"Internal":"false",
"Labels":"com.docker.compose.network=tripleperformance_traefik,com.docker.compose.project=tripleperformance_prod,com.docker.compose.version=1.29.2",
"Name":"tripleperformance_traefik",
"Scope":"local"}

Here you can see that the docker compose version was 1.29.2 and not at all 2.20.2 as it should have been. The reason was that somehow, there was still an old “docker-compose” script in /usr/bin that has been used to recreate the stack.

So here are the steps required to repair the stack:

  1. replace the content of /usr/bin/docker-compose with docker compose --compatibility "$@"
  2. run docker compose down to stop your stack completely
  3. run docker system prune to remove the network
  4. run docker compose up -d to start your stack again

Hope it helps

Perhaps the correct way here is to refer to the default network not by its name but by its object key default?

yes indeed, the networks attribute in service definition refers to keys. By the way you can use net1 if you prefer this over default, just don’t use 2 entries to define a single network

@sd-evo docker network prune

@Scherebart earlier version didn’t checked network labels and as a result may select a network “by name” that is not the expected one (name unicity is not enforced by docker engine) resulting in weird bugs

Before downgrading Docker, you may try this solution. It works for me.

At my docker-compose YAML files, I’m using:

networks:
  myname:
    external: true

I create the network manually, with docker network create myname beforehand. Then, I can use all the YAML files as modules (I start containers that share this network with docker compose -f something.yaml up -d ).

Note: If I ever shut all containers down, the network stays. You’d need to remove it manually if you want to…

@tflavin default did not work for me but using the network name did.

@AnjaLiebermann If you still need it, the accepted answer here outlines the process of how I downgraded on macOS: https://stackoverflow.com/questions/62217678/can-i-roll-back-to-a-previous-version-of-docker-desktop

I downloaded and installed version 4.20.0, and that seems to be doing exactly what I need to get my container fleet running again. Yay!