build-push-action: Build Push action failing with GHA cache on self-hosted runner

Troubleshooting

Behaviour

The action fails at the caching step.

The abort message is “Github actions is temporarily unavailable”, but status shows no issues with github actions. Additionally, other steps in the build process work just fine. This issue is still here after several days.

Also potentially relevant: I am using a self-hosted runner on pop-os

The relevant logs are here:

#20 exporting cache
#20 preparing build cache for export
#20 preparing build cache for export 44.4s done
#20 writing layer sha256:30e1b5776710e06922c72408b2d8e6ac56f1cf66b7d97697c8c28eafe4047ba8
#20 writing layer sha256:30e1b5776710e06922c72408b2d8e6ac56f1cf66b7d97697c8c28eafe4047ba8 0.5s done
#20 writing layer sha256:371803ae1c4092d5f537dcd5d83325e87d3f137ee2b5a827a9ad4c553c6dadde
#20 writing layer sha256:371803ae1c4092d5f537dcd5d83325e87d3f137ee2b5a827a9ad4c553c6dadde 0.2s done
#20 writing layer sha256:6603222a2d2bc0adbeb6f130df205a6d879ef0e45e2a6fb1362e5549eac7f5a1
#20 writing layer sha256:6603222a2d2bc0adbeb6f130df205a6d879ef0e45e2a6fb1362e5549eac7f5a1 0.2s done
#20 writing layer sha256:6e03d496f5213fdc7b85bafc814df0743309719ffa366c78da9b33f4ab19cfe4
#20 writing layer sha256:6e03d496f5213fdc7b85bafc814df0743309719ffa366c78da9b33f4ab19cfe4 0.2s done
#20 writing layer sha256:83098237b6d3febc7584c1f16076a32ac01def85b0d220ab46b6ebb2d6e7d4d4
#20 writing layer sha256:83098237b6d3febc7584c1f16076a32ac01def85b0d220ab46b6ebb2d6e7d4d4 0.0s done
#20 writing layer sha256:88a1d963e569e91cd91639d93f6df2bdb1e64b7cd394003a5d8bb54ff231c11e
#20 writing layer sha256:88a1d963e569e91cd91639d93f6df2bdb1e64b7cd394003a5d8bb54ff231c11e 0.2s done
#20 writing layer sha256:93bbf21dbbeec1bbdb76eb6bd4ff5a2bceae29fb2b8bc287a13b8572f9c23c03
#20 writing layer sha256:93bbf21dbbeec1bbdb76eb6bd4ff5a2bceae29fb2b8bc287a13b8572f9c23c03 60.2s done
#20 ERROR: error writing layer blob: failed to parse error response 503: { 'message': 'GitHub Actions is temporarily unavailable. Please visit https://www.githubstatus.com/ for the status of our services.', 'ref': 'Ref A: 96CC8A3C83314F12BEA57B8D983F48B6 Ref B: ASHEDGE1321 Ref C: 2021-12-08T15:48:22Z' }: invalid character '\'' looking for beginning of object key string
------
 > exporting cache:
------
error: failed to solve: error writing layer blob: failed to parse error response 503: { 'message': 'GitHub Actions is temporarily unavailable. Please visit https://www.githubstatus.com/ for the status of our services.', 'ref': 'Ref A: 96CC8A3C83314F12BEA57B8D983F48B6 Ref B: ASHEDGE1321 Ref C: 2021-12-08T15:48:22Z' }: invalid character '\'' looking for beginning of object key string
Error: buildx failed with: error: failed to solve: error writing layer blob: failed to parse error response 503: { 'message': 'GitHub Actions is temporarily unavailable. Please visit https://www.githubstatus.com/ for the status of our services.', 'ref': 'Ref A: 96CC8A3C83314F12BEA57B8D983F48B6 Ref B: ASHEDGE1321 Ref C: 2021-12-08T15:48:22Z' }: invalid character '\'' looking for beginning of object key string

Is there any issue with using gha cache in build-push-action on a self-hosted runner? If so, can we add some documentation of this breakage and how to work-around it? Maybe a better error message should be added as well.

Steps to reproduce this issue

1. Run action with GHA cache on ubuntu-latest
2. Verify that the action does not fail
3. Re-run action with GHA cache on [self-hosted, linux]
4. Verify that the action does fail

Expected behaviour

Action should not fail

Actual behaviour

Action failed

Configuration

• Repository URL (if public): private
• Build URL (if public): private

# action.yml
name: Test and Build
on:
  pull_request:
  push: 
    branches: [main]

jobs:
  build_and_test_and_push:
    runs-on: [self-hosted, linux]
    steps:
      - uses: actions/checkout@v2
        with:
          submodules: recursive

      - name: Set up Docker Buildx
        id: buildx
        uses: docker/setup-buildx-action@v1
        with:
          install: true
      -
        name: Login to DockerHub
        uses: docker/login-action@v1 
        with:
          username: ${{ secrets.DOCKER_USER }}
          password: ${{ secrets.DOCKER_PASS }}

      - name: Test Build
        uses: docker/build-push-action@master
        with:
          builder: ${{ steps.buildx.outputs.name }}
          context: .
          file: ${{ secrets.DOCKER_TEST_FILE }}
          push: false
          cache-from: type=gha
          cache-to: type=gha,mode=max

      - name: Build production image
        uses: docker/build-push-action@master
        with:
          builder: ${{ steps.buildx.outputs.name }}
          file: ${{ secrets.DOCKER_FILE }}
          context: .
          push: true
          tags: ${{ secrets.DOCKER_USER }}/${{ secrets.DOCKER_REPO }}:latest
          platforms: linux/amd64,linux/arm64,linux/arm/v7,linux/arm/v6
          cache-from: type=gha
          cache-to: type=gha,mode=max

Logs

logs_119.zip