compose: Docker Compose doesn't work when SSH connection used to remote Docker Engine
Description of the issue
Just trying out the new SSH connection introduced in Docker 18.09 and I noticed an error when attempting to do docker-compose up
whilst targeting a remote Docker Engine instance.
Errors message below appears to indicate that Compose isn’t aware of the SSH protocol for this purpose
docker.errors.DockerException: Invalid bind address protocol: ssh://xfoxy.secinternal.local
[486] Failed to execute script docker-compose
Context information (for bug reports)
Output of docker-compose version
docker-compose version 1.23.1, build b02f1306
docker-py version: 3.5.0
CPython version: 3.6.7
OpenSSL version: OpenSSL 1.1.0f 25 May 2017
Output of docker version
Client:
Version: 18.09.0
API version: 1.39
Go version: go1.10.4
Git commit: 4d60db4
Built: Wed Nov 7 00:49:01 2018
OS/Arch: linux/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 18.09.0
API version: 1.39 (minimum version 1.12)
Go version: go1.10.4
Git commit: 4d60db4
Built: Wed Nov 7 00:16:44 2018
OS/Arch: linux/amd64
Experimental: false
Output of docker-compose config
(Make sure to add the relevant -f
and other flags)
networks:
testnet: {}
services:
dradis:
image: raesene/dradis
networks:
testnet: null
ports:
- 3000/tcp
volumes:
- data:/data:rw
sectest:
image: raesene/sectest
networks:
testnet: null
ports:
- 22/tcp
volumes:
- data:/data:rw
version: '3.0'
volumes:
data: {}
Steps to reproduce the issue
- Configure a Docker client (18.09) to connect to a remote Docker engine instance via SSH
- Run
docker-compose up
in a directory with a docker-compose.yml file. - Error occors.
Observed result
Error occurs
Expected result
Docker compose contacts the remote docker engine instance to create the containers.
Stacktrace / full error message
Traceback (most recent call last):
File "bin/docker-compose", line 6, in <module>
File "compose/cli/main.py", line 71, in main
File "compose/cli/main.py", line 124, in perform_command
File "compose/cli/command.py", line 42, in project_from_options
File "compose/cli/command.py", line 123, in get_project
File "compose/cli/command.py", line 94, in get_client
File "compose/cli/docker_client.py", line 127, in docker_client
File "site-packages/docker/api/client.py", line 118, in __init__
File "site-packages/docker/utils/utils.py", line 256, in parse_host
docker.errors.DockerException: Invalid bind address protocol: ssh://xfoxy.secinternal.local
[486] Failed to execute script docker-compose
Additional information
Client is WSL (Ubuntu 18.04) Server is Ubuntu 18.04 running Docker 18.09.
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Reactions: 8
- Comments: 35
Any update on this?
Why is this issue closed? I am still unable to set a docker-compose context to a docker engine running on a remote server over ssh.
I dug into this a bit today. Compose via SSH only works if you:
user:password@host
), orssh-agent
Compose does not support the
IdentityFile
parameter in your.ssh/config
. This is an easy fix that I made locally; I can submit a PR if there’s enough interest.Any update ? Docker-compose works through docker context and ssh-agent but not with ssh_config
Any news on this? I’m running this: docker-compose -H ssh://user@host down and I get the following:
Traceback (most recent call last): File “docker-compose”, line 6, in <module> File “compose\cli\main.py”, line 71, in main File “compose\cli\main.py”, line 124, in perform_command File “compose\cli\command.py”, line 42, in project_from_options File “compose\cli\command.py”, line 123, in get_project File “compose\cli\command.py”, line 94, in get_client File “compose\cli\docker_client.py”, line 127, in docker_client File “site-packages\docker\api\client.py”, line 166, in init File “site-packages\docker\transport\sshconn.py”, line 84, in init File “site-packages\docker\transport\sshconn.py”, line 94, in _connect File “site-packages\paramiko\client.py”, line 437, in connect File “site-packages\paramiko\client.py”, line 676, in _auth File “site-packages\paramiko\agent.py”, line 379, in init File “site-packages\paramiko\agent.py”, line 65, in _connect File “site-packages\paramiko\agent.py”, line 83, in _send_message File “site-packages\paramiko\agent.py”, line 91, in _read_all paramiko.ssh_exception.SSHException: lost ssh-agent [16740] Failed to execute script docker-compose
Tested this with docker-compose version 1.24.0-rc1 but it doesn’t look like it is reading my
~/.ssh/config
.DOCKER_HOST=ssh://my.docker.host docker ps
works fine whileDOCKER_HOST=ssh://my.docker.host docker-compose ps
gives meparamiko.ssh_exception.AuthenticationException: Authentication failed
.still an issue: (base) home@J django-on-docker % DOCKER_HOST=ssh://name@domain.com docker-compose -f docker-compose.staging.yml up -d --build Traceback (most recent call last): File “docker-compose”, line 6, in <module> File “compose/cli/main.py”, line 72, in main File “compose/cli/main.py”, line 125, in perform_command File “compose/cli/command.py”, line 76, in project_from_options File “compose/cli/command.py”, line 142, in get_project File “compose/cli/docker_client.py”, line 47, in get_client File “compose/cli/docker_client.py”, line 174, in docker_client File “site-packages/docker/api/client.py”, line 166, in init File “site-packages/docker/transport/sshconn.py”, line 111, in init File “site-packages/docker/transport/sshconn.py”, line 119, in _connect File “site-packages/paramiko/client.py”, line 349, in connect File “site-packages/paramiko/util.py”, line 283, in retry_on_signal File “site-packages/paramiko/client.py”, line 349, in <lambda> TimeoutError: [Errno 60] Operation timed out [61197] Failed to execute script docker-compose
Yep. Looks like
docker-compose
does not use ~/.ssh/config when using DOCKER_HOST variable, whiledocker
doesIn the meantime, this workaround works for me:
Launch socat in a terminal:
Point compose to local port 8888
Any update?
settings sshd config to MaxSessions 30 helped as workaround
same problem here. On local context everything works like a charm. When I try to launch the docker-compose on a remote host via ssh the docker engine on the stops and restarts the services but doesn’t update it. command “trace”
I believe that most of the SSH related problems stem from the fact that
docker-py
(which is used bydocker-compose
) uses theparamiko
ssh library instead of defaulting to thessh
binary like thedocker
cli.There’s an issue in
docker-py
discussing this: https://github.com/docker/docker-py/issues/2289Can confirm I am having the same issue as @raesene
I spent two days on this but… I got it working on GitLab CI
Run the ssh-agent, add the SSH private key from a file to the agent with the
-k
flag. And most importantlyCOMPOSE_PARAMIKO_SSH=1
this will use SSH client from compose version 1.27.x (i hope i remember correctly) and voila, it works.Confirm that using
docker-compose -H ssh://user@my.host up
gives me:But with docker command
docker -H ssh://user@docker.host ps
works normally, using docker-compose version 1.24.1, build 4667896b on Mac OS and Docker version 19.03.1, build 74b1e89Support for the SSH protocol will be added in the next version of Compose. https://github.com/docker/docker-py/issues/2159