cache: Permission denied creating tar for Apt files

Based on @joshmgross’s latest comment in https://github.com/actions/cache/issues/133#issuecomment-629381394, here’s my use case, same as @evandrocoan https://github.com/actions/cache/issues/133#issuecomment-602695437…

caching Apt list and package files in Ubuntu

Example workflow

The Apt step here takes about 35 seconds without caching. I’m hoping the cache will improve that. I’m also not 100% sure about the paths because I haven’t been able to test it yet.

      - name: Use Apt lists cache
        uses: actions/cache@v1
        with:
          path: /var/lib/apt/lists
          key: ${{ runner.os }}-apt-lists
      - name: Use Apt packages cache
        uses: actions/cache@v1
        with:
          path: /var/cache/apt
          key: ${{ runner.os }}-apt-packages
      - name: Install Apt dependencies
        run: |
          sudo apt-get update
          sudo apt-get install -y liblua5.1-dev luarocks

Error during post job

Post job cleanup.
/bin/tar -cz -f /home/runner/work/_temp/207e89c2-b0e5-4443-915a-2eafc37bc59b/cache.tgz -C /var/cache/apt .
/bin/tar: ./archives/partial: Cannot open: Permission denied
/bin/tar: ./archives/lock: Cannot open: Permission denied
/bin/tar: Exiting with failure status due to previous errors
[warning]Tar failed with error: The process '/bin/tar' failed with exit code 2

About this issue

Original URL
State: open
Created 4 years ago
Reactions: 15
Comments: 22 (5 by maintainers)

Commits related to this issue

Switch over to using `nix` flakes It seems flakes are going to take over the world, so we're going to try them out for this. The thing that seems actually interesting is that since flakes require th... — committed to joneshf/terraform-provider-openwrt by joneshf a year ago
Switch over to using `nix` flakes It seems flakes are going to take over the world, so we're going to try them out for this. The thing that is actually interesting about this change is that since fl... — committed to joneshf/terraform-provider-openwrt by joneshf a year ago
Revert "GHA: platform-linux: Adds /etc/ansible/facts.d to the caching path list" This reverts commit 900a45335b912913fdba109d196e5eaf85920914. Refs: <https://github.com/actions/cache/issues/324> — committed to EA31337/EA-Tester by kenorb 2 months ago

Most upvoted comments

I was able to at least successfully save a cache of the downloaded .deb packages with this step:

      - name: Cache APT packages
        uses: actions/cache@v2
        with:
          path: |
            /var/cache/apt/archives/**.deb
            !/var/cache/apt/archives/partial
            !/var/cache/apt/archives/lock
          key: ${{ runner.os }}-apt

However, when attempting to restore this cache, the attempt to extract these cached files still results in Cannot open: Permission denied errors for each file:

Received 0 of 85361645 (0.0%), 0.0 MBs/sec
Received 71303168 of 85361645 (83.5%), 34.0 MBs/sec
Received 85361645 of 85361645 (100.0%), 35.8 MBs/sec
Cache Size: ~81 MB (85361645 B)
/bin/tar --use-compress-program zstd -d -xf /home/runner/work/_temp/6cb673f4-802b-4627-a69c-fc30b46f495f/cache.tzst -P -C /home/runner/work/haxe/haxe
/bin/tar: ../../../../../var/cache/apt/archives/autopoint_0.19.8.1-6ubuntu0.3_all.deb: Cannot open: Permission denied
/bin/tar: ../../../../../var/cache/apt/archives/bubblewrap_0.2.1-1ubuntu0.1_amd64.deb: Cannot open: Permission denied
 ⋮
 ⋮ (50 more lines omitted)
 ⋮
/bin/tar: ../../../../../var/cache/apt/archives/x11proto-randr-dev_2018.4-4_all.deb: Cannot open: Permission denied
/bin/tar: ../../../../../var/cache/apt/archives/x11proto-xinerama-dev_2018.4-4_all.deb: Cannot open: Permission denied
/bin/tar: Exiting with failure status due to previous errors
Warning: Tar failed with error: The process '/bin/tar' failed with exit code 2

If we could just have an option for running the /bin/tar command as sudo, that would make this tool significantly easier to use for caching system packages (like those installed by apt).

AJMansfield on Feb 4, 2021

I fixed a similar issue by setting the restore target’s permissions with chmod -R a+rwx before the caching step. The restore seems to action correctly with this, but it for sure would be better not to have to do this.

This took a 16min build down to 1min, because a full compile step for a utility binary could be skipped; no need to install the language’s runtime, clone the remote repo, then build and install the binary.

Saving time matters for GH Actions. Anything that can be done to reduce the time for them to complete, even if it’s only a few seconds, can have a big impact.

OldhamMade on Jan 29, 2021

I have a workflow that opens a Nix shell. It has a lot of small dependencies, and it takes about 4 minutes to download them from various Nix binary caches. Seems like it would be faster to restore from github’s cache. Is there some reason the restore can’t just run as root?

chris-martin on Jul 2, 2021

FWIW, I saw that this action seems to offer caching for apt installs: https://github.com/awalsh128/cache-apt-pkgs-action.

A quick look at that seems to indicate that it tries to cache the package files after installation, which seems extremely risky. I’d like to simply cache the results of apt-get update (~12s on the ubuntu-22.04 runner) and the actual .deb files downloaded (~9 seconds for libgdal-dev and its dependencies), to speed that up a bit.

l0b0 on Jan 17, 2023

This issue is stale because it has been open for 200 days with no activity. Leave a comment to avoid closing this issue in 5 days.

github-actions[bot] on May 7, 2022

It has nothing to do with the problem itself, but you should think a little more about caching. Downloading and decompressing are also costly. Considering those, is 35 seconds really long? In my opinion, it’s not.

smorimoto on May 26, 2020

I wonder if anyone tried messing the file permissions on the host before caching occurs, so the current user can rewrite the system cache. It is bit of security nightmare but it might work and it would happen only on CI.

ssbarnea on Jun 27, 2022

Caching system dependencies installed using apt is a a no brainer when it comes to use cases. If you have to run sudo apt-get install ... stands to reason you need to run sudo tar ... to restore the cache.

This feature would be very useful to a lot of people! 👍

michaeltryby on Aug 13, 2021

In my opinion, there is basically no need to run an update. It’s probably faster if you don’t do it. Also, the package manager is not a build system; it probably won’t be slower than you think. (Unless you specify that explicitly.)

smorimoto on May 26, 2020

And the current setup doesn’t guarantee that the contents of /var/lib/apt/lists will not be changed.

smorimoto on May 26, 2020