buildx: host-gateway no longer supported in docker build

This is really annoying. Using a hardcoded ip won’t work as it can change. For me it is 172.18.0.1 and for others 172.17.0.1. The only workaround which might work would be to add an environment variable in the docker host which is regularly updated and reference that one.

I’m facing this same issue, using docker 24.0.1 on Ubuntu. I noticed it when I tried to build a docker-compose with extra_hosts on the build section. I need to access a postgres sever that is running on the host machine. So I’m using extra_hosts “host.docker.internal:host-gateway” to make the connection.

— Addendum

I’m running on Linux. So I changed extra_hosts to point to the docker-host IP and things worked. Here is what I placed on my docker-compose:

extra_hosts:
            - "host.docker.internal:172.17.0.1"

source: https://forums.docker.com/t/how-to-reach-localhost-on-host-from-docker-container/113321/4

Also affected by this. Presumably there is no test coverage for this feature, since it doesn’t work in 23.0.0 and 23.0.1?

Ok it seems to be a bug, I was able to repro on current 23.0.1:

$ docker build --add-host=host.docker.internal:host-gateway .
[+] Building 0.0s (0/0)                                                                                                                                                                                                                       
ERROR: failed to solve: failed to parse additional hosts: failed to parse IP host-gateway

I will take a look.

fwiw, problem is still present in 24.0.1 for me:

$ docker version
Client: Docker Engine - Community
 Version:           24.0.1
 API version:       1.43
 Go version:        go1.20.4
 Git commit:        6802122
 Built:             Fri May 19 18:06:21 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.1
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.4
  Git commit:       463850e
  Built:            Fri May 19 18:06:21 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Then:

ERROR: failed to solve: failed to parse additional hosts: failed to parse IP host-gateway

Thank you @thaJeztah for the explanation. That all makes sense.

That is unfortunate that this feature is broken at the moment. Can we confirm that this is a bug that is effecting multiple people and not just me? This will help me decide whether I should focus on debugging my system/package configuration on ubuntu or into looking for a temporary workaround.

Your work is much appreciated!

Yes, those pull requests were meant to fix issues around this (and are part of the 23.0 release). It’s possible though that other changes are still needed to either BuildKit, or buildx (which provides the client for docker build), or the Dockefile front-end (parser).

Could I get some clarification on what the moby/moby project is really and how it relates to docker?

Docker uses a client/server model with an API. Docker Consists of various components. On a Linux installation of Docker Engine;

• The docker/cli (this repository) repository contains the source code for the command-line client (docker). There’s additional plugins installed that handle specific parts of the CLI (docker compose, docker build and docker buildx).
• The moby/moby repository contains the source-code used to build the Docker Engine (the dockerd “daemon”), which provides the API, and contains the server-side components to run your containers, pull images, and build images.
• For building images, the dockerd daemon contains the “builder”, which is code embedded from the moby/buildkit repository.
• In addition, there’s further “lower level” components, such as containerd (OCI intermediate runtime), and runc (OCI runtime)

All of the above is Open Source. If you run Docker Desktop, there’s additional components and features, some of which are not open source, which includes a VM and LinuxKit Linux distribution (to allow you to run Linux containers on macOS and Windows, and provide additional security boundaries), but also includes a GUI / Dashboard, Management tools, Extensions, etc.

Awesome! Thanks for sharing! I forgot about the release notes document. I’ll definitely refer to that doc next time!

You can also review https://docs.docker.com/engine/release-notes when you update, and look for the fix there 😉

Ah @neersighted beat me to it 😅

Looks like fixes went into 24.0.3 https://github.com/moby/moby/pull/45790

And buildx v0.11.1 through https://github.com/docker/buildx/pull/1926

You need both updated to at least those versions to get the fix

The fix requires buildx v0.11.1, and daemon v24.0.3 (which were released together). The version you have provided is the CLI, which does not need to be updated to get this fix; however I find it unlikely that you have the correct daemon and buildx version given that CLI version.

Decent solution, but I can’t guarantee all the containers I’m running are on the same docker network bridge. They’d have different IPs. We do not wish to hardcode these.

I agree. It was just a quick fix that I used. But by no means a definitive solution.

I still have the same issue as well.

(base) ➜  kubernetes git:(master) ✗ docker version
Client: Docker Engine - Community
 Version:           24.0.1
 API version:       1.43
 Go version:        go1.20.4
 Git commit:        6802122
 Built:             Fri May 19 18:06:21 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.1
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.4
  Git commit:       463850e
  Built:            Fri May 19 18:06:21 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

It seems https://github.com/docker/buildx/pull/1446 and https://github.com/moby/moby/pull/44537 are not enough. The error failed to parse additional hosts comes from BuildKit when build is forwarded through the gateway: https://github.com/moby/buildkit/blob/v0.10/frontend/dockerfile/builder/build.go#L122.