zarf: image pull fail from corrupted Crane cache

Describe what should be investigated or refactored

Seeing a flake in the test-external workflow. Images are failing to be saved.

Workflow run Relevant logs:

  •  Fetching info for 9 images. This step may take several seconds to complete.
  •  Fetched info for 9 images
  •  Pulling 9 images (0.00 Byte of 243.74 MBs)

 WARNING  Failed to save images in parallel, falling back to sequential save: All attempts fail:
          #1: error writing layer: expected blob size 3419706, but only wrote 3207362
          #2: error writing layer: expected blob size 3419706, but only wrote 3207362
     ERROR:  failed to create package: All attempts fail:
             #1: error writing layer: expected blob size 3419706, but only wrote 3207362
             #2: error writing layer: expected blob size 3419706, but only wrote 3207362
    common.go:33: 
        	Error Trace:	/home/runner/work/zarf/zarf/src/test/external/common.go:33
        	            				/home/runner/work/zarf/zarf/src/test/external/ext_in_cluster_test.go:165

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 2
  • Comments: 15 (12 by maintainers)

Most upvoted comments

I believe the temporary solution while we work on a permanent fix in #3559 is to either use zarf tools clear-cache before running zarf package create or set the --zarf-cache directory to a random temporary directory each run.

@AustinAbro321 would you be open to updating the title and description on this issue to reflect the more widespread issue? At first glance, this looks like a simple test flake but it’s the issue that tracks the widespread issue of the crane cache expected blob size blah, but only wrote blah stuff which is the reason for replacing crane.

EDIT: one of the reasons for replacing crane

It happens so persistently for me that I ended up doing this pretty much anywhere I’m making zarf packages now. https://github.com/defenseunicorns-partnerships/wfapi/blob/main/scripts/build_zarf_package.sh