setup-buildx-action: kubernetes driver fails to configure multi-node setup due to missing `node` option

Behaviour

Steps to reproduce this issue

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
        with:
          driver: kubernetes
          platforms: linux/amd64
          driver-opts: |
            "namespace=buildkit"
            "nodeselector=""kubernetes.io/arch=amd64"""
          append: |
            - platforms: linux/arm64
              driver-opts:
                - '"namespace=buildkit"'
                - '"nodeselector=""kubernetes.io/arch=arm64"""'

Expected behaviour

Create a working builder with an arm64 and an amd64 node

Actual behaviour

Creating a new builder instance
  /usr/bin/docker buildx create --name builder-ec5ab75e-cc9a-4e71-b560-72962afcfd54 --driver kubernetes --driver-opt "namespace=buildkit" --driver-opt "nodeselector=""kubernetes.io/arch=arm64""" --buildkitd-flags --allow-insecure-entitlement security.insecure --allow-insecure-entitlement network.host --platform linux/amd64 --use
  builder-ec5ab75e-cc9a-4e71-b560-72962afcfd54
Appending node(s) to builder
  /usr/bin/docker buildx create --name builder-ec5ab75e-cc9a-4e71-b560-72962afcfd54 --append --driver-opt "namespace=buildkit" --driver-opt "nodeselector=""kubernetes.io/arch=arm64""" --platform linux/arm64
  ERROR: invalid duplicate endpoint kubernetes:///builder-ec5ab75e-cc9a-4e71-b560-72962afcfd54?deployment=&kubeconfig=

Workaround

this seems to be working:


      - name: Set up Docker Buildx
        id: setup-builder
        run: |
          UUID=$(uuidgen)
          BUILDER=$(docker buildx create --name builder-${UUID} --node builder-amd64-${UUID} --driver kubernetes --driver-opt "namespace=buildkit" --driver-opt "nodeselector=""kubernetes.io/arch=amd64""" --platform linux/amd64 --use)
          docker buildx create --name builder-${UUID} --node builder-arm64-${UUID} --append --driver-opt "namespace=buildkit" --driver-opt "nodeselector=""kubernetes.io/arch=arm64""" --platform linux/arm64
          docker buildx inspect
          echo "name=${BUILDER}" >> $GITHUB_OUTPUT

      - name: Docker meta
        id: meta
        uses: docker/metadata-action@v4
        with:
          images: ghcr.io/.../...
          tags: |
            type=ref,event=branch
            type=ref,event=pr
            type=semver,pattern={{version}}
            type=semver,pattern={{major}}.{{minor}}

      - name: Login to GitHub Container Registry
        uses: docker/login-action@v2
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Build and push
        id: push
        uses: docker/build-push-action@v4
        with:
          builder: ${{ steps.setup-builder.outputs.name }}
          context: .
          cache-from: type=gha
          cache-to: type=gha,mode=max
          file: build/package/Dockerfile
          push: ${{ github.event_name != 'pull_request' }}
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          platforms: linux/amd64,linux/arm64

      - name: Clean up Docker Buildx
        if: ${{ always() }}
        run: |
          docker buildx rm ${{ steps.setup-builder.outputs.name }}

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 22 (11 by maintainers)

Commits related to this issue

Most upvoted comments

Awesome. I will open a follow-up to fix this issue upstream on buildx repo.

Indeed with both builders in append, and a name supplied this works now: https://github.com/overmindtech/aws-source/pull/131/commits/b44996aadd303f4f0f881e1fe75d00bd6f7fb5f1