poetry: POETRY_CACHE_DIR or cache-dir config are not used at all.

I am on the latest Poetry version.
I have searched the issues of this repo and believe that this is not a duplicate.
If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

OS version and name: Windows 10
Poetry version: 1.0.5
Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/mdgilene/0da1568f8ac90615e89cd1331115d80a

Issue

Setting the cache-dir location using either the environment variable POETRY_CACHE_DIR or with poetry config cache-dir /path/to/new/cache does not work. Poetry still uses the default location C:\Users\<user>\AppData\Local\pypoetry\Cache.

This will not work for me as there are size constraints on my user profile directory. Beacuse of this I need to relocate this directory to another drive.

I believe this to be cause by the fact that the environment/config settings are never read when determining the cache location.

poetry/utils/appdirs.py

def user_cache_dir(appname):
    r"""
    Return full path to the user-specific cache dir for this application.
        "appname" is the name of application.
    Typical user cache directories are:
        macOS:      ~/Library/Caches/<AppName>
        Unix:       ~/.cache/<AppName> (XDG default)
        Windows:    C:\Users\<username>\AppData\Local\<AppName>\Cache
    On Windows the only suggestion in the MSDN docs is that local settings go
    in the `CSIDL_LOCAL_APPDATA` directory. This is identical to the
    non-roaming app data dir (the default returned by `user_data_dir`). Apps
    typically put cache data somewhere *under* the given dir here. Some
    examples:
        ...\Mozilla\Firefox\Profiles\<ProfileName>\Cache
        ...\Acme\SuperApp\Cache\1.0
    OPINION: This function appends "Cache" to the `CSIDL_LOCAL_APPDATA` value.
    """
    if WINDOWS:
        # Get the base path
        path = os.path.normpath(_get_win_folder("CSIDL_LOCAL_APPDATA"))

        # Add our app name and Cache directory to it
        path = os.path.join(path, appname, "Cache")
    elif sys.platform == "darwin":
        # Get the base path
        path = expanduser("~/Library/Caches")

        # Add our app name to it
        path = os.path.join(path, appname)
    else:
        # Get the base path
        path = os.getenv("XDG_CACHE_HOME", expanduser("~/.cache"))

        # Add our app name to it
        path = os.path.join(path, appname)

    return path

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 13
Comments: 23 (12 by maintainers)

Commits related to this issue

Add caching to poetry install process. - This is the documented way to provide a cache location to poetry,though it does not actually speed up the build significantly. - See https://github.com/pyth... — committed to paketo-buildpacks/poetry-install by robdimsdale 2 years ago
Initial poc (#2) * Initial functionality - No offline functionality as Poetry does not current support vendoring - Caching may work but is untested * Add .github directory. * Fix lint in te... — committed to paketo-buildpacks/poetry-install by robdimsdale 2 years ago

Most upvoted comments

You may need to set three environment variables to alter poetry’s default directories instead of single POETRY_CACHE_DIR

Oh interesting…

Why is it that I need to set all 3?
Why doesn’t simply using poetry config to set the values and have them stored in the config.toml suffice? Why do I also need to set environment variables?

adam-grant-hendry on Jun 18, 2022

Can confirm this happens in version 1.1.13 in a docker container:

FROM python:3.9-slim-buster

ENV \
    POETRY_HOME="/opt/poetry" \
    POETRY_CACHE_DIR="/opt/poetry/.cache" \
    POETRY_VERSION=1.1.13 \
    # Poetry config
    POETRY_NO_INTERACTION=true \
    POETRY_NO_DEV=true \
    POETRY_VIRTUALENVS_CREATE=false \
    POETRY_VIRTUALENVS_IN_PROJECT=false \
    POETRY_EXPERIMENTAL_NEW_INSTALLER=false 

CMD ["bash"]

Firing up the container with user 1000 and running poetry lock fails with this error:

$ docker-compose run --rm --user=1000:1000 my-service bash
$ poetry lock

Skipping virtualenv creation, as specified in config file.
Updating dependencies
Resolving dependencies... (0.0s)

  LockFailed

  failed to create /.cache/pypoetry/cache/repositories/pypi/_http/2/3/2/5/7/25a703cfdb14-dbb50740.82384136640494608341

It fails because ofc user 1000 does not have access to /.cache, why is it trying to create a directory there?

arielnmz on Mar 2, 2022

I’m sorry but “just remove it” is not a valid solution. It is common practice among many development tools that utilize a local cache to allow the user to configure where this cache is stored.

My workplace uses network storage for users home directories so that settings can be used on multiple machines. This however means that we are VERY limited on how much can be stored on the user profile (<500Mb). I HAVE to move this cache to another location or I just simply cannot use the tool.

mdgilene on Jan 12, 2021

Resovled by #5672.

abn on May 24, 2022

Is there any workaround for this on windows? Perhaps some environment variable like XDG_CACHE_DIR I can set?

I’m running multiple instances of Poetry at one once and I’m getting a conflict in the cache directory.

  [Errno 17] File exists: 'C:\\Windows\\system32\\config\\systemprofile\\AppData\\Local\\pypoetry\\Cache\\cache\\repositories\\pypi\\_http\\4\\8\\8\\4\\4\\48844afbd98c85dd6f00d0efdc493230a4f783137c66108a422a9d24'
  at f:\gitlab-runner\builds\c1fMCazL\1\james.duley\python-template\.poetry\venv\lib\site-packages\cachecontrol\caches\file_cache.py:46 in _secure_open_write

So I want to use a different cache directory for each invocation. Although, this is probably a separate issue that needs to be addressed.

parched on Aug 5, 2021

only artifacts directory goes into POETRY_CACHE_DIR, but cache\repositories\pypi generated in %LOCALAPPDATA%

I can confirm this is still happening in Linux. Even after setting POETRY_CACHE_DIR, I see ~/.cache/pypoetry/cache/repositories/pypi/ created with contents.

Poetry Version: 1.1.4

mohan43u on Jan 6, 2021

Far from ideal, but as a workaround you can temporarily set/replace XDG_CACHE_HOME, eg:

XDG_CACHE_HOME=/path/to/another/drive/cache poetry add black

(tested on 1.0.9)

pwoolvett on Aug 8, 2020

I don’t think POETRY_HOME is the right thing to leverage for this, because it’s surprising that it would override the XDG_CACHE_HOME default behaviour, and also I’d expect nothing to change in POETRY_HOME except when using the self-updater (and then only if Poetry was installed by get-poetry.py).

The PR doesn’t modify POETRY_HOME dir, it creates hidden directories under POETRY_HOME and do the change under those hidden directories.

This way, all I have to do in my bootstrap script is set POETRY_HOME to one location under the project’s root directory, so that everything poetry creates (including get-poetry.py) contained within the project’s root directory.

On the last comment, I’m not sure what poetry shell has to do with this. If I’m changing XDG_CACHE_HOME, of course I’m changing it for everything. That’s the whole point.

the point is why someone has to change XDG_CACHE_HOME in the first place in order to make poetry store its cache and virtualenv dirs in a different place.

Also, there is no way to make poetry store its config in a different place other than changing XDG_CONFIG_HOME.

In my openion, XDG_*_HOME dirs are fallback dirs when the user not explicitly specify where to store the files. But poetry is using XDG_*_HOME dirs assuming user is not going to care about providing the locations.

mohan43u on Jan 12, 2021