buildx: [output clipped, log limit 1MiB reached] and then exit code 7

I’m using a regular docker build, how can I increase the log limit?

Also hit by this crap with normal docker build 😦

Even --progress=plain doesn’t help… This is insane!! I just can’t get my full log and see the compiler error!

for convenience, here’s a bash one-liner: docker buildx build . --builder "$(docker buildx create --driver-opt env.BUILDKIT_STEP_LOG_MAX_SIZE=10000000 --driver-opt env.BUILDKIT_STEP_LOG_MAX_SPEED=10000000)"

I have the docker 20.10.11 And it seems that the only working solution is to add this variable DOCKER_BUILDKIT=0 before running docker:

DOCKER_BUILDKIT=0 docker build

This will prevent using new build functionalities

For those who use buildx github actions for image building, just need extra with to prevent this:

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v1
        with:
          driver-opts: env.BUILDKIT_STEP_LOG_MAX_SIZE=10485760

I’m struggling with this myself, running a buildkit enabled docker build from a linux command line - pulling hairs on how to pass those params… can anyone please point me to the right direction? (docker does not accept --driver-opt, and restarting dockerd with BUILDKIT_STEP_LOG_MAX_SIZE=-1 and BUILDKIT_STEP_LOG_MAX_SPEED=-1 environment vars doesn’t have any effect).

@OmryAr So far I haven’t seen any good resolution in docker. Adding env var to docker daemon via systemd doesn’t help much as that somehow introduced a random hanging issue for large log, even when it’s only a few MB. We never had this issue until the release of this feature, or without setting those env var.

And there’s no “unlimited” value.

I think a reasonable fix is to disable clipping by default, or at least have a reliable kill switch.

Hi @yardenshoham ! After researching this quite deeply, I can try to explain this:

• At the time of this writing, log limits cannot be arbitrarily set using normal params to the docker build command.
• To set log limits globally, you need set appropriate environment params for the dockerd service, and restart it for the configuration to take effect. Assuming a systemd based Linux-distro server, you’ll need to add the following lines to /etc/sysconfig/docker: BUILDKIT_STEP_LOG_MAX_SIZE=10000000 BUILDKIT_STEP_LOG_MAX_SPEED=10000000
• Alternatively you might create a builder instance and set it up with your desired options, as shown in my comment above.

HTH!

clipping can be disabled with buildx create --driver-opt env.BUILDKIT_STEP_LOG_MAX_SIZE=-1 --driver-opt env.BUILDKIT_STEP_LOG_MAX_SPEED=-1 but this should have no effect on the outcome of the build. If it does then please post a reproducer.

increase log limits from 1 MiB to at least 10MiB.

for 10MB --driver-opt env.BUILDKIT_STEP_LOG_MAX_SIZE=10485760

The true question is: how is this problem supposed to be solved? There must be a way to solve that, right? Maybe extending the max log size or even setting it to “unlimited”.

[output clipped, log limit 1MiB reached]

The clipping issue has been creating big trouble for us for long. It doubles the work of debugging because none of the meaningful error can be captured within the limited size. All the error has to be repro by running Dockerfile commands manually within docker run. And any one-off errors (e.g. network issues/race conditions/environment specific issues/…) is simply not reproducible.

However, even when we try to raise the limit with env var for dockerd (BUILDKIT_STEP_LOG_MAX_SIZE and BUILDKIT_STEP_LOG_MAX_SPEED set to 1~2G via systemd), it will just hang in the middle of build like this:

The docker build will hang there with partial log for hours in this case.

clipping can be disabled with buildx create --driver-opt env.BUILDKIT_STEP_LOG_MAX_SIZE=-1 --driver-opt env.BUILDKIT_STEP_LOG_MAX_SPEED=-1 but this should have no effect on the outcome of the build. If it does then please post a reproducer.

[output clipped, log limit -1B reached]

This worked on Windows I added DOCKER_BUILDKIT with value 0 to the system environment variables, then ran Docker Desktop 4.12.0, when I do a build the log keeps going, running all the way with no limit or hang ups.

I have the docker 20.10.11 And it seems that the only working solution is to add this variable DOCKER_BUILDKIT=0 before running docker:

DOCKER_BUILDKIT=0 docker build

This will prevent using new build functionalities

This really work! thanks!

This is a fairly sad regression. Unable to view more than 1MB of logs on our own machine?

Here’s my stats:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.6.3-docker)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 9
  Running: 1
  Paused: 0
  Stopped: 8
 Images: 25
 Server Version: 20.10.8
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.47-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 15.64GiB
 Name: docker-desktop

Fixed by https://github.com/moby/buildkit/pull/1934

Thanks @jessehui ! So after a long while of digging into the docs and experimenting - I’ve managed to create a buildx instance set with these configuration params. Something on the lines of:

docker buildx create --driver-opt env.BUILDKIT_STEP_LOG_MAX_SIZE=1000000,env.BUILDKIT_STEP_LOG_MAX_SPEED=100000000 --use

So now my revised question would be - how can I configure the default instance (without creating another one - which I wasn’t intending to… and seems wasteful)?

• When running docker buildx build, out-of-the-box, I am running the default instance, right?
• How can I pass --driver-opts to the default instance?

I have same issue. I want to how to stop logging or increase limits.

I am facing the same issue, Can someone tell me how to stop logging or increase log limits from 1 MiB to at least 10MiB.

Any way to increase the log limit cache size?