skaffold: Skaffold does not work when using remote docker over ssh

Expected behavior

Using skaffold in an environment where DOCKER_HOST=ssh://somehost is defined should work, as this is a supported docker transport method.

Actual behavior

Skaffold tries to connect over http to the given host, ignoring the ‘ssh://’ part

Information

  • Skaffold version: v1.22.0
  • Operating system: Windows
  • Installed via: Scoop

Steps to reproduce the behavior

  1. Run docker on a remote host that you can also connect with ssh
  2. Set environment variable DOCKER_HOST=ssh://yourhost
  3. Run skaffold dev on a project

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 17 (6 by maintainers)

Most upvoted comments

You can forward socket file using ssh and -L option. For example:

ssh -L $HOME/remote-docker.sock:/var/run/docker.sock user@docker-remote-host.example.com

Where /var/run/docker.sock is the dockerd socket file on remote machine docker-remote-host.example.com

Then you can use this local socket to forward queries to remote dockerd:

DOCKER_HOST=unix://$HOME/remote-docker.sock skaffold dev

Add -H tcp://<your-remote-ip> to /usr/lib/systemd/system/docker.service, and use env DOCKER_HOST=tcp://<your-remote-ip>:2375 skaffold build -v debug instead of skaffold build -v debug, it’s work to me.

It seems to have something to do with this

Maybe need a PR

As you may know, running docker with tcp basically GIVES ANYONE WHO CAN REACH THIS PORT ROOT ACCESS TO THE MACHINE. https://docs.docker.com/go/attack-surface/

That being said:

On the remote machine:

Add an override to your systemd docker configuration:

sudo systemctl edit docker

This will open the editor, edit it so the first lines look like:

### Editing /etc/systemd/system/docker.service.d/override.conf
### Anything between here and the comment below will become the new contents of the file

[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2375 --containerd=/run/containerd/containerd.sock

### Lines below this comment will be discarded

Restart your docker service on the remote machine:

sudo systemctl restart docker

On your machine:

Let’s say the remote machine is 192.168.0.20:

docker context create remote \
   --default-stack-orchestrator=swarm \
   --docker host=tcp://192.168.0.20:2375

Now, when you want to use the remote machine, switch context:

docker context use remote

When you want to use local docker:

docker context use default

Using skaffold with the remote docker:

Unfortunately, skaffold won’t recognize the context docker is in, so you still will have to inform that in a environment variable.

DOCKER_HOST=tcp://192.168.0.20:2375 skaffold build

Add -H tcp://<your-remote-ip> to /usr/lib/systemd/system/docker.service, and use env DOCKER_HOST=tcp://<your-remote-ip>:2375 skaffold build -v debug instead of skaffold build -v debug, it’s work to me.

It seems to have something to do with this

Maybe need a PR

I believe I’m getting the same issue. Here’s the actual error:

getting imageID for <docker image>: error during connect: Get "http://<username>%40<ssh host>/v1.24/images/<docker image>/json": dial tcp: lookup <username>@<ssh host>: no such host. Docker build ran into internal error. Please retry.
If this keeps happening, please open an issue..

Im getting this error with all combinations of build.local.useBuildkit and build.local.useDockerCLI