moby: [DOCKER SWARM 1.13]Docker overlay not working with encryption

Description Hi, we are now evaluating the impact of docker 1.13 over our application Our application is composed of several nodejs microservices and mongodb in replicaset mode. For this evaluation we are executing some load tests, simulating 1400 concurrent users. Everything worked fine until we enabled the encryption of the overlay network ๐Ÿ˜ฆ it seems that services are not able to reach each other anymore!

To make it easier to debug for you, we have found a simple way to reproduce the issue

Steps to reproduce the issue:

  1. create 2 networks overlay ; one with encryption enabled and the second one without the encryption:
docker network create --driver overlay  --subnet 192.168.1.0/24 --attachable --opt encrypted encrypted_network
docker network create --driver overlay  --subnet 192.168.1.0/24 --attachable clear_network
  1. create a image using the following docker file:
FROM ubuntu:14.04
RUN apt-get update
RUN apt-get -y install wget
  1. Deploy the image on 2 different hosts using the not encrypted network with the command : docker run --network clear_network -t -i <IMAGE_NAME> /bin/bash

  2. start a netcat server listening on port 12345 in one of the containers : nc -l 12345

  3. Send text using netcat from the second container using the command : echo "some text"| nc 192.168.1.3 12345 The IP is the one of the first container

  4. The text is sent from the second to the first container. If you repeat these steps using the encrypted overlay network it will not work ๐Ÿ˜ฆ

About this issue

  • Original URL
  • State: open
  • Created 7 years ago
  • Comments: 25 (10 by maintainers)

Most upvoted comments

hello, everyone, has anyone been able to fix this? Iโ€™m facing the same problem. trying to deploy nginx on swarm and got the following output:

"starting container failed: cannot join secure network: encryption keys not present"

itโ€™s a small swarm, 5 hosts (1 manager, 4 workers), iโ€™m having this issue with 2 workers.

Thank Alessandro. I have just enabled the protocol 50 on my ubuntu servers and it works fine now ๐Ÿ˜ƒ

@fabio-barile From your description, it looks like packet send over the encrypted network is a complete blackhole. If that is the case, please make sure your environment allows the ESP packets to be exchanged across hosts. See https://docs.docker.com/engine/swarm/swarm-tutorial/#open-ports-between-the-hosts.