buildkit: ERROR: failed to solve: DeadlineExceeded: context deadline exceeded

Sorry for the vague error message.

We have a docker image that runs some things. Most times, it runs fine. Occasionally, it errors at the very end with a vague error message that is not from the yml file but I think is from the docker image exiting/ending. (All our commands run fine in the container. It is failing after the last one completes).

We run the image from a github action, if that matter.

The question is:

What is not being solved?
What deadline is exceeded?
Can it be increased?
Is there some way of running the container in debug mode for more information?

Thanks in advanced.

2023-10-04T20:39:13.3582748Z #24 exporting to client directory
2023-10-04T20:39:14.5610603Z #24 copying files 28B
2023-10-04T20:39:19.7302573Z #24 copying files 1.75GB 5.2s
2023-10-04T20:39:24.6910750Z #24 copying files 3.81GB 10.2s
2023-10-04T20:39:29.7983597Z #24 copying files 5.89GB 15.3s
2023-10-04T20:39:35.6174985Z #24 copying files 7.41GB 21.1s
2023-10-04T20:39:49.8824772Z #24 copying files 7.69GB 35.4s
2023-10-04T20:39:54.8880765Z #24 copying files 10.02GB 40.4s
2023-10-04T20:40:04.0740589Z #24 copying files 10.52GB 49.4s done
2023-10-04T20:40:04.0741210Z #24 DONE 50.7s
2023-10-04T20:41:33.3139725Z ERROR: failed to solve: DeadlineExceeded: context deadline exceeded
2023-10-04T20:41:33.3615053Z ##[error]Process completed with exit code 1.
2023-10-04T20:41:33.4182323Z Cleaning up orphan processes

[epasveer@ws7-01 ~]$ docker --version
Docker version 20.10.8, build 3967b7d
[epasveer@ws7-01 ~]$

About this issue

Original URL
State: open
Created 9 months ago
Reactions: 2
Comments: 28 (7 by maintainers)

Most upvoted comments

I’m facing a similar issue on my Ubuntu 20.04.6 LTS servers. My os and docker/builkdit config:

# uname -r
5.4.0-167-generic
# docker version
Client: Docker Engine - Community
 Version:           24.0.7
 API version:       1.43
 Go version:        go1.20.10
 Git commit:        afdd53b
 Built:             Thu Oct 26 09:08:01 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.7
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.10
  Git commit:       311b9ff
  Built:            Thu Oct 26 09:08:01 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.25
  GitCommit:        d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f
 runc:
  Version:          1.1.10
  GitCommit:        v1.1.10-0-g18a0cb0
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

# docker buildx ls
NAME/NODE                          DRIVER/ENDPOINT             STATUS   BUILDKIT             PLATFORMS
buildx-docker-container-nodejs *   docker-container                                          
  buildx-docker-container-nodejs0  unix:///var/run/docker.sock running  f84cfe3              linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/amd64/v4, linux/386
# docker buildx inspect buildx-docker-container-nodejs
Name:          buildx-docker-container-nodejs
Driver:        docker-container
Last Activity: 2023-12-14 15:15:07 +0000 UTC

Nodes:
Name:           buildx-docker-container-nodejs0
Endpoint:       unix:///var/run/docker.sock
Driver Options: image="moby/buildkit:nightly"
Status:         running
Flags:          --debug --oci-worker-gc=false
Buildkit:       f84cfe3
Platforms:      linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/amd64/v4, linux/386
Labels:
 org.mobyproject.buildkit.worker.executor:         oci
 org.mobyproject.buildkit.worker.hostname:         277c9036902a
 org.mobyproject.buildkit.worker.network:          host
 org.mobyproject.buildkit.worker.oci.process-mode: sandbox
 org.mobyproject.buildkit.worker.selinux.enabled:  false
 org.mobyproject.buildkit.worker.snapshotter:      overlayfs

I disabled garbage collection and enabled debug on buildkit level. My Dockerfile:

# syntax=docker/dockerfile:1.6
ARG NODE_VERSION=18.18.2
FROM bitnami/node:${NODE_VERSION} as base

FROM base as npm-packages
WORKDIR /src
COPY npm-packages/package.json npm-packages/package-lock.json npm-packages/.npmrc npm-packages/
COPY npm-packages/utils/package.json npm-packages/utils/package-lock.json npm-packages/utils/.npmrc npm-packages/utils/
WORKDIR /src/npm-packages
RUN npm ci --progress=false --no-audit
WORKDIR /src/npm-packages/utils
RUN npm ci --progress=false --no-audit

# Common builder - ensure same RUN / COPY is included in other Web builds
FROM npm-packages as builder
WORKDIR /src
COPY Some.Application/package.json Some.Application/package-lock.json Some.Application/
WORKDIR /src/Some.Application
RUN npm ci --progress=false --no-audit --ignore-scripts

FROM builder as build
WORKDIR /src/npm-packages
COPY npm-packages/ ./
WORKDIR /src/Some.Application
COPY Some.Application/ ./
RUN npm run build
WORKDIR /src/npm-packages/utils
RUN npm prune --production

FROM scratch as publishresults
COPY --from=build /src/Some.Application/package.json /output/
COPY --from=build /src/Some.Application/node_modules/ /output/node_modules/
COPY --from=build /src/Some.Application/target/Some.Application/ /output/target/
COPY --from=build /src/Some.Application/target/npm-packages/ /output/npm-packages/
COPY --from=build /src/npm-packages/utils/node_modules/ /output/npm-packages/utils/node_modules/

FROM base as final
WORKDIR /app
VOLUME /var/log/someapplication/chat
COPY --from=publishresults --link=true /output/ /app/
CMD [ "node", "target/index.js" ]

I use below command to build it:

docker --debug buildx build --progress=plain --builder buildx-docker-container-nodejs --target publishresults --output type=local,dest=/var/lib/Some.Application_release_1.2 --cache-from type=registry,ref=registry.gitlab.com/someapplication/buildkit-cache:tag --file Some.Application/Dockerfile .

Stack trace/error:

16:15:08  #0 building with "buildx-docker-container-nodejs" instance using docker-container driver
16:15:08                                                                                      
16:15:08  #1 [internal] load build definition from Dockerfile                                                                                                                               16:15:08  #1 transferring dockerfile: 1.64kB done                                                                                                                                           
16:15:08  #1 DONE 0.0s                                                                        
16:15:08                                                                                                                                                                                    
16:15:08  #2 resolve image config for docker.io/docker/dockerfile:1.6                         
16:15:08  #2 DONE 0.2s                                                                        
16:15:08                
16:15:08  #3 docker-image://docker.io/docker/dockerfile:1.6@sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021
16:15:08  #3 resolve docker.io/docker/dockerfile:1.6@sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021 done                                                           
16:15:08  #3 CACHED
16:15:08  
16:15:08  #4 [internal] load metadata for docker.io/bitnami/node:18.18.2
16:15:08  #4 DONE 0.2s
16:15:08  
16:15:08  #5 [internal] load .dockerignore
16:15:08  #5 transferring context: 832B done
16:15:08  #5 DONE 0.0s
16:15:08  
16:15:08  #6 [internal] load build context
16:15:08  #6 DONE 0.0s
16:15:08  
16:15:08  #7 importing cache manifest from registry.gitlab.com/someapplicationco/someapplication/buildkit-cache:tag
16:15:08  #7 ...
16:15:08  
16:15:08  #8 [auth] someapplicationco/someapplication/buildkit-cache:pull token for registry.gitlab.com
16:15:08  #8 DONE 0.0s
16:15:09  
16:15:09  #7 importing cache manifest from registry.gitlab.com/someapplicationco/someapplication/buildkit-cache:tag
16:15:09  #7 inferred cache manifest type: application/vnd.oci.image.index.v1+json done
16:15:09  #7 DONE 0.9s
16:15:09  
16:15:09  #9 [base 1/1] FROM docker.io/bitnami/node:18.18.2@sha256:479f965ef6bca556db4dcf8cd25a0408a11788c305a149f21f6eaaabaae9d0de
16:15:09  #9 resolve docker.io/bitnami/node:18.18.2@sha256:479f965ef6bca556db4dcf8cd25a0408a11788c305a149f21f6eaaabaae9d0de done
16:15:09  #9 DONE 0.0s
16:15:09  
16:15:09  #6 [internal] load build context
16:15:09  #6 transferring context: 14.38kB 0.0s done
16:15:09  #6 DONE 0.0s
16:15:09  
16:15:09  #10 [build 3/5] WORKDIR /src/Some.Application
16:15:09  #10 CACHED
16:15:09  
16:15:09  #11 [npm-packages 1/7] WORKDIR /src
16:15:09  #11 CACHED
16:15:09  
16:15:09  #12 [publishresults 2/6] COPY --from=build /src/Some.Application/node_modules/ /output/node_modules/
16:15:09  #12 CACHED
16:15:09  
16:15:09  #13 [npm-packages 3/7] COPY npm-packages/utils/package.json npm-packages/utils/package-lock.json npm-packages/utils/.npmrc npm-packages/utils/
16:15:09  #13 CACHED
16:15:09  
16:15:09  #14 [build 1/5] WORKDIR /src/npm-packages
16:15:09  #14 CACHED
16:15:09  
16:15:09  #15 [publishresults 1/6] COPY --from=build /src/Some.Application/package.json /output/
16:15:09  #15 CACHED
16:15:09  
16:15:09  #16 [publishresults 5/6] COPY --from=build /src/npm-packages/node_modules/ /output/npm-packages/node_modules/
16:15:09  #16 CACHED
16:15:09  
16:15:09  #17 [npm-packages 7/7] RUN npm ci --progress=false --no-audit
16:15:09  #17 CACHED
16:15:09  
16:15:09  #18 [npm-packages 5/7] RUN npm ci --progress=false --no-audit
16:15:09  #18 CACHED
16:15:09  
16:15:09  #19 [build 4/5] COPY Some.Application/ ./
16:15:09  #19 CACHED
16:15:09  
16:15:09  #20 [builder 1/4] WORKDIR /src
16:15:09  #20 CACHED
16:15:09  
16:15:09  #21 [build 5/5] RUN npm run build
16:15:09  #21 CACHED
16:15:09  
16:15:09  #22 [builder 3/4] WORKDIR /src/Some.Application
16:15:09  #22 CACHED
16:15:09  
16:15:09  #23 [builder 4/4] RUN npm ci --progress=false --no-audit --ignore-scripts
16:15:09  #23 CACHED
16:15:09  
16:15:09  #24 [builder 2/4] COPY Some.Application/package.json Some.Application/package-lock.json Some.Application/
16:15:09  #24 CACHED
16:15:09  
16:15:09  #25 [publishresults 3/6] COPY --from=build /src/Some.Application/target/Some.Application/ /output/target/
16:15:09  #25 CACHED
16:15:09  
16:15:09  #26 [npm-packages 4/7] WORKDIR /src/npm-packages
16:15:09  #26 CACHED
16:15:09  
16:15:09  #27 [build 2/5] COPY npm-packages/ ./
16:15:09  #27 CACHED
16:15:09  
16:15:09  #28 [npm-packages 6/7] WORKDIR /src/npm-packages/utils
16:15:09  #28 CACHED
16:15:09  
16:15:09  #29 [publishresults 4/6] COPY --from=build /src/Some.Application/target/npm-packages/ /output/npm-packages/
16:15:09  #29 CACHED
16:15:09  
16:15:09  #30 [npm-packages 2/7] COPY npm-packages/package.json npm-packages/package-lock.json npm-packages/.npmrc npm-packages/
16:15:09  #30 CACHED
16:15:09  
16:15:09  #31 [publishresults 6/6] COPY --from=build /src/npm-packages/utils/node_modules/ /output/npm-packages/utils/node_modules/
16:15:09  #31 CACHED
16:15:09  
16:15:09  #32 exporting to client directory
16:15:09  #32 copying files 26B
16:16:48  Cancelling nested steps due to timeout
16:16:48  Sending interrupt signal to process
16:17:00  DEBUG: using config store "/home/jenkins/.docker/buildx" based in "$BUILDX_CONFIG" environment variable
16:17:00  DEBUG: using config store "/home/jenkins/.docker/buildx" based in "$BUILDX_CONFIG" environment variable
16:17:00  DEBUG: using config store "/home/jenkins/.docker/buildx" based in "$BUILDX_CONFIG" environment variable
16:17:00  ERROR: failed to solve: Canceled: context canceled
16:17:00  12809 v0.11.2 /usr/libexec/docker/cli-plugins/docker-buildx --debug buildx build --progress=plain --builder buildx-docker-container-nodejs --target publishresults --output type=>
16:17:00  google.golang.org/grpc.getChainUnaryInvoker.func1
16:17:00        google.golang.org/grpc@v1.53.0/clientconn.go:361
16:17:00  go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc.UnaryClientInterceptor.func1
16:17:00        go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc@v0.40.0/interceptor.go:99
16:17:00  github.com/moby/buildkit/client.filterInterceptor.func1
16:17:00        github.com/moby/buildkit@v0.12.1-0.20230717122532-faa0cc7da353/client/client.go:384
16:17:00  google.golang.org/grpc.chainUnaryClientInterceptors.func1
16:17:00        google.golang.org/grpc@v1.53.0/clientconn.go:349
16:17:00  google.golang.org/grpc.(*ClientConn).Invoke
16:17:00        google.golang.org/grpc@v1.53.0/call.go:35
16:17:00  github.com/moby/buildkit/api/services/control.(*controlClient).Solve
16:17:00        github.com/moby/buildkit@v0.12.1-0.20230717122532-faa0cc7da353/api/services/control/control.pb.go:2208
16:17:00  github.com/moby/buildkit/client.(*Client).solve.func2
16:17:00        github.com/moby/buildkit@v0.12.1-0.20230717122532-faa0cc7da353/client/solve.go:258
16:17:00  golang.org/x/sync/errgroup.(*Group).Go.func1
16:17:00        golang.org/x/sync@v0.2.0/errgroup/errgroup.go:75
16:17:00  runtime.goexit
16:17:00        runtime/asm_amd64.s:1598
16:17:00  
16:17:00  12809 v0.11.2 /usr/libexec/docker/cli-plugins/docker-buildx --debug buildx build --progress=plain --builder buildx-docker-container-nodejs --target publishresults --output type=>
16:17:00  github.com/moby/buildkit/client.(*Client).solve.func2
16:17:00        github.com/moby/buildkit@v0.12.1-0.20230717122532-faa0cc7da353/client/solve.go:273
16:17:00  golang.org/x/sync/errgroup.(*Group).Go.func1
16:17:00        golang.org/x/sync@v0.2.0/errgroup/errgroup.go:75
16:17:00  runtime.goexit
16:17:00        runtime/asm_amd64.s:1598

On buildkit container level I get:

  1 time="2023-12-14T15:15:03Z" level=info msg="auto snapshotter: using overlayfs"
  2 time="2023-12-14T15:15:03Z" level=warning msg="using host network as the default"
  3 time="2023-12-14T15:15:03Z" level=info msg="found worker \"j15zbih8yphnky0k6rxkrt7y1\", labels=map[org.mobyproject.buildkit.worker.executor:oci org.mobyproject.buildkit.worker.hostname:277c9036902a org.mobyproject.buildkit.worker.network:host org.mobyproject.buildkit.worker.oci.process-mode:sandbox org.mobyproject.buildkit.worker.selinux.enabled:false org.mobyproject.buil    dkit.worker.snapshotter:overlayfs], platforms=[linux/amd64 linux/amd64/v2 linux/amd64/v3 linux/amd64/v4 linux/386]"
  4 time="2023-12-14T15:15:03Z" level=warning msg="skipping containerd worker, as \"/run/containerd/containerd.sock\" does not exist"
  5 time="2023-12-14T15:15:03Z" level=info msg="found 1 workers, default=\"j15zbih8yphnky0k6rxkrt7y1\""
  6 time="2023-12-14T15:15:03Z" level=warning msg="currently, only the default worker can be used."
  7 time="2023-12-14T15:15:03Z" level=info msg="running server on /run/buildkit/buildkitd.sock"
  8 time="2023-12-14T15:15:03Z" level=debug msg="session started" spanID=ae2c1997099d20cd traceID=91f68e5bdcd180ee13f0079e4900519c
  9 time="2023-12-14T15:15:03Z" level=debug msg="session finished: <nil>" spanID=ae2c1997099d20cd traceID=91f68e5bdcd180ee13f0079e4900519c
 10 time="2023-12-14T15:15:03Z" level=debug msg="session started" spanID=24f672a9b7c04609 traceID=91f68e5bdcd180ee13f0079e4900519c
 11 time="2023-12-14T15:15:03Z" level=debug msg="reusing ref for local: djdi820tpz8jd0decn318qbgi" span="[internal] load build definition from Dockerfile" spanID=9696eba11e679f6c traceID=91f68e5bdcd180ee13f0079e4900519c
 12 time="2023-12-14T15:15:03Z" level=debug msg="diffcopy took: 3.428898ms" span="[internal] load build definition from Dockerfile" spanID=fcda1758417b0573 traceID=91f68e5bdcd180ee13f0079e4900519c
 13 time="2023-12-14T15:15:03Z" level=debug msg=resolving host=registry-1.docker.io
 14 time="2023-12-14T15:15:03Z" level=debug msg="do request" host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=buildkit/v0.0.0+unknown request.method=    HEAD url="https://registry-1.docker.io/v2/docker/dockerfile/manifests/1.6"
 15 time="2023-12-14T15:15:03Z" level=debug msg="fetch response received" host=registry-1.docker.io response.header.content-length=160 response.header.content-type=application/json response.header.date="Thu, 14 Dec 2023 15:15:03 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=159.100.245.171 response.header.strict-tran    sport-security="max-age=31536000" response.header.www-authenticate="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:docker/dockerfile:pull\"" response.status="401 Unauthorized" url="https://registry-1.docker.io/v2/docker/dockerfile/manifests/1.6"
 16 time="2023-12-14T15:15:03Z" level=debug msg=Unauthorized header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:docker/dockerfile:pull\"" host=registry-1.docker.io
 17 time="2023-12-14T15:15:03Z" level=debug msg="do request" host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=buildkit/v0.0.0+unknown request.method=    HEAD url="https://registry-1.docker.io/v2/docker/dockerfile/manifests/1.6"
 18 time="2023-12-14T15:15:04Z" level=debug msg="fetch response received" host=registry-1.docker.io response.header.content-length=8404 response.header.content-type=application/vnd.oci.image.index.v1+json response.header.date="Thu, 14 Dec 2023 15:15:04 GMT" response.header.docker-content-digest="sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021" response    .header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=159.100.245.171 response.header.etag="\"sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021\"" response.header.strict-transport-security="max-age=31536000" response.status="200 OK" url="https://registry-1.docker.io/v2/docker/dockerfile/manifests/1.6"
 19 time="2023-12-14T15:15:04Z" level=debug msg=resolved desc.digest="sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021" host=registry-1.docker.io
 20 time="2023-12-14T15:15:04Z" level=debug msg=fetch digest="sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021" mediatype=application/vnd.oci.image.index.v1+json size=8404
 21 time="2023-12-14T15:15:04Z" level=debug msg=fetch digest="sha256:657fcc512c7369f4cb3d94ea329150f8daf626bc838b1a1e81f1834c73ecc77e" mediatype=application/vnd.oci.image.manifest.v1+json size=482
 22 time="2023-12-14T15:15:04Z" level=debug msg=fetch digest="sha256:a17ee7fff8f5e97b974f5b48f51647d2cf28d543f2aa6c11aaa0ea431b44bb89" mediatype=application/vnd.oci.image.config.v1+json size=1271
 23 time="2023-12-14T15:15:04Z" level=debug msg=fetch digest="sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021" mediatype=application/vnd.oci.image.index.v1+json size=8404
 24 time="2023-12-14T15:15:04Z" level=debug msg=fetch digest="sha256:657fcc512c7369f4cb3d94ea329150f8daf626bc838b1a1e81f1834c73ecc77e" mediatype=application/vnd.oci.image.manifest.v1+json size=482
 25 time="2023-12-14T15:15:04Z" level=debug msg=fetch digest="sha256:a17ee7fff8f5e97b974f5b48f51647d2cf28d543f2aa6c11aaa0ea431b44bb89" mediatype=application/vnd.oci.image.config.v1+json size=1271
 26 time="2023-12-14T15:15:04Z" level=debug msg="load cache for docker-image://docker.io/docker/dockerfile:1.6@sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021 with j15zbih8yphnky0k6rxkrt7y1::rfmjjkt9dppczm0jyqyq65s8e"
 27 time="2023-12-14T15:15:04Z" level=debug msg="serving grpc connection" spanID=edda0dcd5efce83e traceID=91f68e5bdcd180ee13f0079e4900519c
 28 time="2023-12-14T15:15:04Z" level=debug msg="> creating yxvjg8oqtn6db6o3u3jkd42e1 [/bin/dockerfile-frontend]" spanID=edda0dcd5efce83e traceID=91f68e5bdcd180ee13f0079e4900519c
 29 time="2023-12-14T15:15:04Z" level=debug msg=resolving host=registry-1.docker.io
 30 time="2023-12-14T15:15:04Z" level=debug msg="do request" host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=buildkit/v0.0.0+unknown request.method=    HEAD url="https://registry-1.docker.io/v2/bitnami/node/manifests/18.18.2"
 31 time="2023-12-14T15:15:04Z" level=debug msg="fetch response received" host=registry-1.docker.io response.header.content-length=155 response.header.content-type=application/json response.header.date="Thu, 14 Dec 2023 15:15:04 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=159.100.245.171 response.header.strict-tran    sport-security="max-age=31536000" response.header.www-authenticate="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:bitnami/node:pull\"" response.status="401 Unauthorized" url="https://registry-1.docker.io/v2/bitnami/node/manifests/18.18.2"
 32 time="2023-12-14T15:15:04Z" level=debug msg=Unauthorized header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:bitnami/node:pull\"" host=registry-1.docker.io
 33 time="2023-12-14T15:15:04Z" level=debug msg="do request" host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=buildkit/v0.0.0+unknown request.method=    HEAD url="https://registry-1.docker.io/v2/bitnami/node/manifests/18.18.2"
 34 time="2023-12-14T15:15:05Z" level=debug msg="fetch response received" host=registry-1.docker.io response.header.content-length=529 response.header.content-type=application/vnd.docker.distribution.manifest.list.v2+json response.header.date="Thu, 14 Dec 2023 15:15:05 GMT" response.header.docker-content-digest="sha256:479f965ef6bca556db4dcf8cd25a0408a11788c305a149f21f6eaaaba    ae9d0de" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=159.100.245.171 response.header.etag="\"sha256:479f965ef6bca556db4dcf8cd25a0408a11788c305a149f21f6eaaabaae9d0de\"" response.header.strict-transport-security="max-age=31536000" response.status="200 OK" url="https://registry-1.docker.io/v2/bitnami/node/manifests/18.    18.2"
 35 time="2023-12-14T15:15:05Z" level=debug msg=resolved desc.digest="sha256:479f965ef6bca556db4dcf8cd25a0408a11788c305a149f21f6eaaabaae9d0de" host=registry-1.docker.io
 36 time="2023-12-14T15:15:05Z" level=debug msg=fetch digest="sha256:479f965ef6bca556db4dcf8cd25a0408a11788c305a149f21f6eaaabaae9d0de" mediatype=application/vnd.docker.distribution.manifest.list.v2+json size=529
 37 time="2023-12-14T15:15:05Z" level=debug msg=fetch digest="sha256:11f03f5ed59b289b810a81c6950b28befcc3b8a3bab5696f4d69c4c9a173a15b" mediatype=application/vnd.docker.distribution.manifest.v2+json size=430
 38 time="2023-12-14T15:15:05Z" level=debug msg=fetch digest="sha256:ea8e91735c2ee3f817e14eda616abf114bd43c1cce28fe6f3df2f51688825413" mediatype=application/vnd.docker.container.image.v1+json size=5785
 39 time="2023-12-14T15:15:05Z" level=debug msg="reusing ref for local: d7v9tu0oqeis7nq9o2x2jc44t" span="[internal] load .dockerignore" spanID=d51631720eb41f9f traceID=91f68e5bdcd180ee13f0079e4900519c
 40 time="2023-12-14T15:15:05Z" level=debug msg="diffcopy took: 3.040671ms" span="[internal] load .dockerignore" spanID=6b782e1e27b8b480 traceID=91f68e5bdcd180ee13f0079e4900519c
 41 time="2023-12-14T15:15:05Z" level=debug msg=resolving host=registry.gitlab.com
 42 time="2023-12-14T15:15:05Z" level=debug msg="do request" host=registry.gitlab.com request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=buildkit/v0.0.0+unknown request.method=H    EAD url="https://registry.gitlab.com/v2/someapplication/someapplication/buildkit-cache/manifests/tag"
 43 time="2023-12-14T15:15:05Z" level=debug msg=fetch digest="sha256:479f965ef6bca556db4dcf8cd25a0408a11788c305a149f21f6eaaabaae9d0de" mediatype=application/vnd.docker.distribution.manifest.list.v2+json size=529
 44 time="2023-12-14T15:15:05Z" level=debug msg=fetch digest="sha256:11f03f5ed59b289b810a81c6950b28befcc3b8a3bab5696f4d69c4c9a173a15b" mediatype=application/vnd.docker.distribution.manifest.v2+json size=430
 45 time="2023-12-14T15:15:05Z" level=debug msg=fetch digest="sha256:ea8e91735c2ee3f817e14eda616abf114bd43c1cce28fe6f3df2f51688825413" mediatype=application/vnd.docker.container.image.v1+json size=5785
 46 time="2023-12-14T15:15:05Z" level=debug msg="fetch response received" host=registry.gitlab.com response.header.content-length=200 response.header.content-type=application/json response.header.date="Thu, 14 Dec 2023 15:15:05 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.gitlab-lb=haproxy-registry-04-lb-gprd response.header.gitlab-sv=regi    stry-us-east1-c response.header.www-authenticate="Bearer realm=\"https://gitlab.com/jwt/auth\",service=\"container_registry\",scope=\"repository:someapplication/someapplication/buildkit-cache:pull\"" response.header.x-content-type-options=nosniff response.status="401 Unauthorized" url="https://registry.gitlab.com/v2/someapplication/someapplication/buildkit-cache/manifests    /tag"
 47 time="2023-12-14T15:15:05Z" level=debug msg=Unauthorized header="Bearer realm=\"https://gitlab.com/jwt/auth\",service=\"container_registry\",scope=\"repository:someapplication/someapplication/buildkit-cache:pull\"" host=registry.gitlab.com
 48 time="2023-12-14T15:15:05Z" level=debug msg="do request" host=registry.gitlab.com request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=buildkit/v0.0.0+unknown request.method=H    EAD url="https://registry.gitlab.com/v2/someapplication/someapplication/buildkit-cache/manifests/tag"
 49 time="2023-12-14T15:15:05Z" level=debug msg="fetch response received" host=registry.gitlab.com response.header.content-length=7021 response.header.content-type=application/vnd.oci.image.index.v1+json response.header.date="Thu, 14 Dec 2023 15:15:05 GMT" response.header.docker-content-digest="sha256:0545f1602be83f3b14ef30e35f9512e408ac873e9ba7d00d2c9571c73f9e74da" response.    header.docker-distribution-api-version=registry/2.0 response.header.etag="\"sha256:0545f1602be83f3b14ef30e35f9512e408ac873e9ba7d00d2c9571c73f9e74da\"" response.header.gitlab-lb=haproxy-registry-04-lb-gprd response.header.gitlab-sv=gke-cny-registry response.header.x-content-type-options=nosniff response.status="200 OK" url="https://registry.gitlab.com/v2/someapplication/so    meapplication/buildkit-cache/manifests/tag"
 50 time="2023-12-14T15:15:05Z" level=debug msg=resolved desc.digest="sha256:0545f1602be83f3b14ef30e35f9512e408ac873e9ba7d00d2c9571c73f9e74da" host=registry.gitlab.com
 51 time="2023-12-14T15:15:05Z" level=debug msg="do request" digest="sha256:0545f1602be83f3b14ef30e35f9512e408ac873e9ba7d00d2c9571c73f9e74da" request.header.accept="application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=buildkit/v0.0.0+unknown request.method=GET url="https://registry.gitlab.com/v2/someapplication/someapplication/buildkit-cache/manifests/sha25    6:0545f1602be83f3b14ef30e35f9512e408ac873e9ba7d00d2c9571c73f9e74da"
 52 time="2023-12-14T15:15:06Z" level=debug msg="fetch response received" digest="sha256:0545f1602be83f3b14ef30e35f9512e408ac873e9ba7d00d2c9571c73f9e74da" response.header.content-length=7021 response.header.content-type=application/vnd.oci.image.index.v1+json response.header.date="Thu, 14 Dec 2023 15:15:06 GMT" response.header.docker-content-digest="sha256:0545f1602be83f3b14e    f30e35f9512e408ac873e9ba7d00d2c9571c73f9e74da" response.header.docker-distribution-api-version=registry/2.0 response.header.etag="\"sha256:0545f1602be83f3b14ef30e35f9512e408ac873e9ba7d00d2c9571c73f9e74da\"" response.header.gitlab-lb=haproxy-registry-04-lb-gprd response.header.gitlab-sv=registry-us-east1-c response.header.x-content-type-options=nosniff response.status="200     OK" url="https://registry.gitlab.com/v2/someapplication/someapplication/buildkit-cache/manifests/sha256:0545f1602be83f3b14ef30e35f9512e408ac873e9ba7d00d2c9571c73f9e74da"
 53 time="2023-12-14T15:15:06Z" level=debug msg="do request" digest="sha256:7b9724a054658bddaf641b36db0724e9794282f9ffec133d8cd2044e5427ecb1" request.header.accept="application/vnd.buildkit.cacheconfig.v0, */*" request.header.user-agent=buildkit/v0.0.0+unknown request.method=GET url="https://registry.gitlab.com/v2/someapplication/someapplication/buildkit-cache/blobs/sha256:7b    9724a054658bddaf641b36db0724e9794282f9ffec133d8cd2044e5427ecb1"
 54 time="2023-12-14T15:15:06Z" level=debug msg="fetch response received" digest="sha256:7b9724a054658bddaf641b36db0724e9794282f9ffec133d8cd2044e5427ecb1" response.header.accept-ranges=bytes response.header.age=2540 response.header.alt-svc="h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000" response.header.cache-control="public,private,max-age=0" response.header.content-leng    th=4038 response.header.content-type=application/octet-stream response.header.date="Thu, 14 Dec 2023 14:32:46 GMT" response.header.etag="\"963f709af13c33eef316f80032de0274\"" response.header.last-modified="Tue, 12 Dec 2023 14:15:17 GMT" response.header.server=UploadServer response.header.x-goog-generation=1702390517080746 response.header.x-goog-hash="crc32c=3nGTiQ==" resp    onse.header.x-goog-hash.1="md5=lj9wmvE8M+7zFvgAMt4CdA==" response.header.x-goog-metageneration=1 response.header.x-goog-storage-class=STANDARD response.header.x-goog-stored-content-encoding=identity response.header.x-goog-stored-content-length=4038 response.header.x-guploader-uploadid=ABPtcPrWm1Id1vk4wATvUMPPAlZiMIohDJSY4WywuUOxNmQzQ_JubzmKcCoWmk4tL_xn-pqimGzf4rGZKK1UBvOM    Gr0qdw response.status="200 OK" url="https://registry.gitlab.com/v2/someapplication/someapplication/buildkit-cache/blobs/sha256:7b9724a054658bddaf641b36db0724e9794282f9ffec133d8cd2044e5427ecb1"
 55 time="2023-12-14T15:15:06Z" level=debug msg="reusing ref for local: n627qinty2c0vgjq94ry42agc" span="[internal] load build context" spanID=119634fce9f7b9f7 traceID=91f68e5bdcd180ee13f0079e4900519c
 56 time="2023-12-14T15:15:06Z" level=debug msg="diffcopy took: 56.040159ms" span="[internal] load build context" spanID=41b1f7a8d4105b3a traceID=91f68e5bdcd180ee13f0079e4900519c
 57 time="2023-12-14T15:15:06Z" level=debug msg="load cache for [build 5/5] RUN npm run build with j15zbih8yphnky0k6rxkrt7y1::p1kw9x7wdk3509e9yvrq5svdo"
 58 time="2023-12-14T15:15:06Z" level=debug msg="session finished: <nil>" spanID=24f672a9b7c04609 traceID=91f68e5bdcd180ee13f0079e4900519c
 59 time="2023-12-14T15:15:07Z" level=debug msg="session started" spanID=431f901070ee549a traceID=0f0c5a1bf5592b6438e1e71b7fe6b2f2
 60 time="2023-12-14T15:15:07Z" level=debug msg="session finished: <nil>" spanID=431f901070ee549a traceID=0f0c5a1bf5592b6438e1e71b7fe6b2f2
 61 time="2023-12-14T15:15:07Z" level=debug msg="session started" spanID=81692ce987c201fa traceID=0f0c5a1bf5592b6438e1e71b7fe6b2f2
 62 time="2023-12-14T15:15:07Z" level=debug msg="reusing ref for local: djdi820tpz8jd0decn318qbgi" span="[internal] load build definition from Dockerfile" spanID=695e084810c43aeb traceID=0f0c5a1bf5592b6438e1e71b7fe6b2f2
 63 time="2023-12-14T15:15:07Z" level=debug msg="diffcopy took: 2.894857ms" span="[internal] load build definition from Dockerfile" spanID=876690d4c3ca1c78 traceID=0f0c5a1bf5592b6438e1e71b7fe6b2f2
 64 time="2023-12-14T15:15:07Z" level=debug msg=resolving host=registry-1.docker.io
 65 time="2023-12-14T15:15:07Z" level=debug msg="do request" host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=buildkit/v0.0.0+unknown request.method=    HEAD url="https://registry-1.docker.io/v2/docker/dockerfile/manifests/1.6"
 66 time="2023-12-14T15:15:08Z" level=debug msg="fetch response received" host=registry-1.docker.io response.header.content-length=8404 response.header.content-type=application/vnd.oci.image.index.v1+json response.header.date="Thu, 14 Dec 2023 15:15:08 GMT" response.header.docker-content-digest="sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021" response    .header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=159.100.245.171 response.header.etag="\"sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021\"" response.header.strict-transport-security="max-age=31536000" response.status="200 OK" url="https://registry-1.docker.io/v2/docker/dockerfile/manifests/1.6"
 67 time="2023-12-14T15:15:08Z" level=debug msg=resolved desc.digest="sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021" host=registry-1.docker.io
 68 time="2023-12-14T15:15:08Z" level=debug msg=fetch digest="sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021" mediatype=application/vnd.oci.image.index.v1+json size=8404
 69 time="2023-12-14T15:15:08Z" level=debug msg=fetch digest="sha256:657fcc512c7369f4cb3d94ea329150f8daf626bc838b1a1e81f1834c73ecc77e" mediatype=application/vnd.oci.image.manifest.v1+json size=482
 70 time="2023-12-14T15:15:08Z" level=debug msg=fetch digest="sha256:a17ee7fff8f5e97b974f5b48f51647d2cf28d543f2aa6c11aaa0ea431b44bb89" mediatype=application/vnd.oci.image.config.v1+json size=1271
 71 time="2023-12-14T15:15:08Z" level=debug msg=fetch digest="sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021" mediatype=application/vnd.oci.image.index.v1+json size=8404
 72 time="2023-12-14T15:15:08Z" level=debug msg=fetch digest="sha256:657fcc512c7369f4cb3d94ea329150f8daf626bc838b1a1e81f1834c73ecc77e" mediatype=application/vnd.oci.image.manifest.v1+json size=482
 73 time="2023-12-14T15:15:08Z" level=debug msg=fetch digest="sha256:a17ee7fff8f5e97b974f5b48f51647d2cf28d543f2aa6c11aaa0ea431b44bb89" mediatype=application/vnd.oci.image.config.v1+json size=1271
 74 time="2023-12-14T15:15:08Z" level=debug msg="load cache for docker-image://docker.io/docker/dockerfile:1.6@sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021 with j15zbih8yphnky0k6rxkrt7y1::rfmjjkt9dppczm0jyqyq65s8e"
 75 time="2023-12-14T15:15:08Z" level=debug msg="serving grpc connection" spanID=e35d2912927b989a traceID=0f0c5a1bf5592b6438e1e71b7fe6b2f2
 76 time="2023-12-14T15:15:08Z" level=debug msg="> creating pllajphfqkt3j8pzknmm879xz [/bin/dockerfile-frontend]" spanID=e35d2912927b989a traceID=0f0c5a1bf5592b6438e1e71b7fe6b2f2
 77 time="2023-12-14T15:15:08Z" level=debug msg=resolving host=registry-1.docker.io
 78 time="2023-12-14T15:15:08Z" level=debug msg="do request" host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=buildkit/v0.0.0+unknown request.method=    HEAD url="https://registry-1.docker.io/v2/bitnami/node/manifests/18.18.2"
 79 time="2023-12-14T15:15:08Z" level=debug msg="fetch response received" host=registry-1.docker.io response.header.content-length=529 response.header.content-type=application/vnd.docker.distribution.manifest.list.v2+json response.header.date="Thu, 14 Dec 2023 15:15:08 GMT" response.header.docker-content-digest="sha256:479f965ef6bca556db4dcf8cd25a0408a11788c305a149f21f6eaaaba    ae9d0de" response.header.docker-distribution-api-version=registry/2.0 response.header.docker-ratelimit-source=159.100.245.171 response.header.etag="\"sha256:479f965ef6bca556db4dcf8cd25a0408a11788c305a149f21f6eaaabaae9d0de\"" response.header.strict-transport-security="max-age=31536000" response.status="200 OK" url="https://registry-1.docker.io/v2/bitnami/node/manifests/18.    18.2"
 80 time="2023-12-14T15:15:08Z" level=debug msg=resolved desc.digest="sha256:479f965ef6bca556db4dcf8cd25a0408a11788c305a149f21f6eaaabaae9d0de" host=registry-1.docker.io
 81 time="2023-12-14T15:15:08Z" level=debug msg=fetch digest="sha256:479f965ef6bca556db4dcf8cd25a0408a11788c305a149f21f6eaaabaae9d0de" mediatype=application/vnd.docker.distribution.manifest.list.v2+json size=529
 82 time="2023-12-14T15:15:08Z" level=debug msg=fetch digest="sha256:11f03f5ed59b289b810a81c6950b28befcc3b8a3bab5696f4d69c4c9a173a15b" mediatype=application/vnd.docker.distribution.manifest.v2+json size=430
 83 time="2023-12-14T15:15:08Z" level=debug msg=fetch digest="sha256:ea8e91735c2ee3f817e14eda616abf114bd43c1cce28fe6f3df2f51688825413" mediatype=application/vnd.docker.container.image.v1+json size=5785
 84 time="2023-12-14T15:15:08Z" level=debug msg="reusing ref for local: d7v9tu0oqeis7nq9o2x2jc44t" span="[internal] load .dockerignore" spanID=242db7e332b717ee traceID=0f0c5a1bf5592b6438e1e71b7fe6b2f2
 85 time="2023-12-14T15:15:08Z" level=debug msg="diffcopy took: 3.75032ms" span="[internal] load .dockerignore" spanID=46bfdf9dfe6dd8cc traceID=0f0c5a1bf5592b6438e1e71b7fe6b2f2
 86 time="2023-12-14T15:15:08Z" level=debug msg=resolving host=registry.gitlab.com
 87 time="2023-12-14T15:15:08Z" level=debug msg="do request" host=registry.gitlab.com request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=buildkit/v0.0.0+unknown request.method=H    EAD url="https://registry.gitlab.com/v2/someapplication/someapplication/buildkit-cache/manifests/tag"
 88 time="2023-12-14T15:15:08Z" level=debug msg="fetch response received" host=registry.gitlab.com response.header.content-length=200 response.header.content-type=application/json response.header.date="Thu, 14 Dec 2023 15:15:08 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.gitlab-lb=haproxy-registry-04-lb-gprd response.header.gitlab-sv=regi    stry-us-east1-c response.header.www-authenticate="Bearer realm=\"https://gitlab.com/jwt/auth\",service=\"container_registry\",scope=\"repository:someapplication/someapplication/buildkit-cache:pull\"" response.header.x-content-type-options=nosniff response.status="401 Unauthorized" url="https://registry.gitlab.com/v2/someapplication/someapplication/buildkit-cache/manifests    /tag"
 89 time="2023-12-14T15:15:08Z" level=debug msg=Unauthorized header="Bearer realm=\"https://gitlab.com/jwt/auth\",service=\"container_registry\",scope=\"repository:someapplication/someapplication/buildkit-cache:pull\"" host=registry.gitlab.com
 90 time="2023-12-14T15:15:08Z" level=debug msg="do request" host=registry.gitlab.com request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=buildkit/v0.0.0+unknown request.method=H    EAD url="https://registry.gitlab.com/v2/someapplication/someapplication/buildkit-cache/manifests/tag"
 91 time="2023-12-14T15:15:09Z" level=debug msg="fetch response received" host=registry.gitlab.com response.header.content-length=7021 response.header.content-type=application/vnd.oci.image.index.v1+json response.header.date="Thu, 14 Dec 2023 15:15:09 GMT" response.header.docker-content-digest="sha256:0545f1602be83f3b14ef30e35f9512e408ac873e9ba7d00d2c9571c73f9e74da" response.    header.docker-distribution-api-version=registry/2.0 response.header.etag="\"sha256:0545f1602be83f3b14ef30e35f9512e408ac873e9ba7d00d2c9571c73f9e74da\"" response.header.gitlab-lb=haproxy-registry-04-lb-gprd response.header.gitlab-sv=registry-us-east1-c response.header.x-content-type-options=nosniff response.status="200 OK" url="https://registry.gitlab.com/v2/someapplication    /someapplication/buildkit-cache/manifests/tag"
 92 time="2023-12-14T15:15:09Z" level=debug msg=resolved desc.digest="sha256:0545f1602be83f3b14ef30e35f9512e408ac873e9ba7d00d2c9571c73f9e74da" host=registry.gitlab.com
 93 time="2023-12-14T15:15:09Z" level=debug msg="do request" digest="sha256:0545f1602be83f3b14ef30e35f9512e408ac873e9ba7d00d2c9571c73f9e74da" request.header.accept="application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=buildkit/v0.0.0+unknown request.method=GET url="https://registry.gitlab.com/v2/someapplication/someapplication/buildkit-cache/manifests/sha25    6:0545f1602be83f3b14ef30e35f9512e408ac873e9ba7d00d2c9571c73f9e74da"
 94 time="2023-12-14T15:15:09Z" level=debug msg="fetch response received" digest="sha256:0545f1602be83f3b14ef30e35f9512e408ac873e9ba7d00d2c9571c73f9e74da" response.header.content-length=7021 response.header.content-type=application/vnd.oci.image.index.v1+json response.header.date="Thu, 14 Dec 2023 15:15:09 GMT" response.header.docker-content-digest="sha256:0545f1602be83f3b14e    f30e35f9512e408ac873e9ba7d00d2c9571c73f9e74da" response.header.docker-distribution-api-version=registry/2.0 response.header.etag="\"sha256:0545f1602be83f3b14ef30e35f9512e408ac873e9ba7d00d2c9571c73f9e74da\"" response.header.gitlab-lb=haproxy-registry-04-lb-gprd response.header.gitlab-sv=registry-us-east1-c response.header.x-content-type-options=nosniff response.status="200     OK" url="https://registry.gitlab.com/v2/someapplication/someapplication/buildkit-cache/manifests/sha256:0545f1602be83f3b14ef30e35f9512e408ac873e9ba7d00d2c9571c73f9e74da"
 95 time="2023-12-14T15:15:09Z" level=debug msg="do request" digest="sha256:7b9724a054658bddaf641b36db0724e9794282f9ffec133d8cd2044e5427ecb1" request.header.accept="application/vnd.buildkit.cacheconfig.v0, */*" request.header.user-agent=buildkit/v0.0.0+unknown request.method=GET url="https://registry.gitlab.com/v2/someapplication/someapplication/buildkit-cache/blobs/sha256:7b    9724a054658bddaf641b36db0724e9794282f9ffec133d8cd2044e5427ecb1"
 96 time="2023-12-14T15:15:09Z" level=debug msg="fetch response received" digest="sha256:7b9724a054658bddaf641b36db0724e9794282f9ffec133d8cd2044e5427ecb1" response.header.accept-ranges=bytes response.header.age=2543 response.header.alt-svc="h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000" response.header.cache-control="public,private,max-age=0" response.header.content-leng    th=4038 response.header.content-type=application/octet-stream response.header.date="Thu, 14 Dec 2023 14:32:46 GMT" response.header.etag="\"963f709af13c33eef316f80032de0274\"" response.header.last-modified="Tue, 12 Dec 2023 14:15:17 GMT" response.header.server=UploadServer response.header.x-goog-generation=1702390517080746 response.header.x-goog-hash="crc32c=3nGTiQ==" resp    onse.header.x-goog-hash.1="md5=lj9wmvE8M+7zFvgAMt4CdA==" response.header.x-goog-metageneration=1 response.header.x-goog-storage-class=STANDARD response.header.x-goog-stored-content-encoding=identity response.header.x-goog-stored-content-length=4038 response.header.x-guploader-uploadid=ABPtcPrWm1Id1vk4wATvUMPPAlZiMIohDJSY4WywuUOxNmQzQ_JubzmKcCoWmk4tL_xn-pqimGzf4rGZKK1UBvOM    Gr0qdw response.status="200 OK" url="https://registry.gitlab.com/v2/someapplication/someapplication/buildkit-cache/blobs/sha256:7b9724a054658bddaf641b36db0724e9794282f9ffec133d8cd2044e5427ecb1"
 97 time="2023-12-14T15:15:09Z" level=debug msg=fetch digest="sha256:479f965ef6bca556db4dcf8cd25a0408a11788c305a149f21f6eaaabaae9d0de" mediatype=application/vnd.docker.distribution.manifest.list.v2+json size=529
 98 time="2023-12-14T15:15:09Z" level=debug msg=fetch digest="sha256:11f03f5ed59b289b810a81c6950b28befcc3b8a3bab5696f4d69c4c9a173a15b" mediatype=application/vnd.docker.distribution.manifest.v2+json size=430
 99 time="2023-12-14T15:15:09Z" level=debug msg=fetch digest="sha256:ea8e91735c2ee3f817e14eda616abf114bd43c1cce28fe6f3df2f51688825413" mediatype=application/vnd.docker.container.image.v1+json size=5785
100 time="2023-12-14T15:15:09Z" level=debug msg="reusing ref for local: n627qinty2c0vgjq94ry42agc" span="[internal] load build context" spanID=ebde5e6ebf0404fe traceID=0f0c5a1bf5592b6438e1e71b7fe6b2f2
101 time="2023-12-14T15:15:09Z" level=debug msg="diffcopy took: 22.524741ms" span="[internal] load build context" spanID=349c7bde32668714 traceID=0f0c5a1bf5592b6438e1e71b7fe6b2f2
102 time="2023-12-14T15:15:09Z" level=debug msg="load cache for [publishresults 6/6] COPY --from=build /src/npm-packages/utils/node_modules/ /output/npm-packages/utils/node_modules/ with j15zbih8yphnky0k6rxkrt7y1::gtbkcolmncytff7u6be9wgxz7"
103 time="2023-12-14T15:16:48Z" level=debug msg="session finished: <nil>" spanID=81692ce987c201fa traceID=0f0c5a1bf5592b6438e1e71b7fe6b2f2

I tried quite a lot of things already:

checked different versions of moby/buildkit image including nightly (due to this comment)
checked different versions of docker daemon
disabled apparmor for docker/whole system
disabled concurrency for buildkit
disabled cache entirely for buildkit

For us the problem happens when as a result of publishresults step we have > 450MB of data to copy/send, it happens in 98% of such cases. For other apps when we have ~100-200MB of data to transfer this problem on same os/docker/buildkit configuration happens very rarely, for ~2% of cases.

Any hints will be really appreciated! 😃

rafalr-ntropy on Dec 14, 2023

i saw @tonistiigi suggesting it before, simple as that:

- docker buildx bake -f docker-bake.hcl
+ docker --debug buildx bake -f docker-bake.hcl

mzihlmann on Oct 19, 2023

@crazy-max iiuc that function isn’t really called in local export as that would call NewFSSyncTargetDir

tonistiigi on Oct 13, 2023