mamba: RuntimeError on cache in multi-users case

I am hitting a similar issue in our setup. We build linux VMs for a users using ansible. This installs mambaforge in /opt and creates an environment with our software installed, this runs as root. The VM is then given to a single user, who will use it for a few days or weeks. We want the user to be able to update the environment. We don’t know who the user will be at install time, so we chmod +w /opt/mambaforge/envs and /opt/mambaforge/pkgs/cache so that whichever user gets the machine they have write access there.

This worked with conda and conda update. But with mamba update we get:

RuntimeError: Operation not permitted: '/mambaforge/pkgs/cache/b556ea5a.json'

I can reproduce with the following docker file:

FROM ubuntu:20.04
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y wget

RUN wget -nv -O Mambaforge.sh https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-$(uname)-$(uname -m).sh
RUN bash Mambaforge.sh -b -p /mambaforge

RUN /mambaforge/bin/mamba init
SHELL ["/bin/bash", "-c"]

RUN source /mambaforge/etc/profile.d/conda.sh && mamba create --yes -n test numpy==1.21.3

RUN chmod -R o+w /mambaforge/envs
RUN chmod -R o+w /mambaforge/pkgs/cache

# Make the cache files look old so that mamba needs to update them
RUN find /mambaforge/pkgs/cache -exec touch -d "7 days ago" {} +

RUN useradd -ms /bin/bash user
USER user

Build with:

docker build --no-cache -t mamba_user_test -f dockerfile .

then in:

docker run -it --rm mamba_user_test bash

run:

source /mambaforge/etc/profile.d/conda.sh && conda activate test && mamba update --yes numpy

 >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "/mambaforge/lib/python3.9/site-packages/conda/exceptions.py", line 1079, in __call__
        return func(*args, **kwargs)
      File "/mambaforge/lib/python3.9/site-packages/mamba/mamba.py", line 917, in exception_converter
        raise e
      File "/mambaforge/lib/python3.9/site-packages/mamba/mamba.py", line 911, in exception_converter
        exit_code = _wrapped_main(*args, **kwargs)
      File "/mambaforge/lib/python3.9/site-packages/mamba/mamba.py", line 869, in _wrapped_main
        result = do_call(args, p)
      File "/mambaforge/lib/python3.9/site-packages/mamba/mamba.py", line 740, in do_call
        exit_code = update(args, parser)
      File "/mambaforge/lib/python3.9/site-packages/mamba/mamba.py", line 630, in update
        return install(args, parser, "update")
      File "/mambaforge/lib/python3.9/site-packages/mamba/mamba.py", line 486, in install
        index = load_channels(pool, channels, repos)
      File "/mambaforge/lib/python3.9/site-packages/mamba/utils.py", line 122, in load_channels
        index = get_index(
      File "/mambaforge/lib/python3.9/site-packages/mamba/utils.py", line 103, in get_index
        is_downloaded = dlist.download(True)
    RuntimeError: Operation not permitted: '/mambaforge/pkgs/cache/ddfd5f96.json'

`$ /mambaforge/condabin/mamba update --yes numpy`

  environment variables:
                 CIO_TEST=<not set>
        CONDA_DEFAULT_ENV=test
                CONDA_EXE=/mambaforge/bin/conda
             CONDA_PREFIX=/mambaforge/envs/test
    CONDA_PROMPT_MODIFIER=(test)
         CONDA_PYTHON_EXE=/mambaforge/bin/python
               CONDA_ROOT=/mambaforge
              CONDA_SHLVL=1
           CURL_CA_BUNDLE=<not set>
                     PATH=/mambaforge/envs/test/bin:/mambaforge/condabin:/usr/local/sbin:/usr/lo
                          cal/bin:/usr/sbin:/usr/bin:/sbin:/bin
       REQUESTS_CA_BUNDLE=<not set>
            SSL_CERT_FILE=<not set>

     active environment : test
    active env location : /mambaforge/envs/test
            shell level : 1
       user config file : /home/user/.condarc
 populated config files : /mambaforge/.condarc
          conda version : 4.10.3
    conda-build version : not installed
         python version : 3.9.7.final.0
       virtual packages : __linux=5.11.0=0
                          __glibc=2.31=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /mambaforge  (read only)
      conda av data dir : /mambaforge/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /mambaforge/pkgs
                          /home/user/.conda/pkgs
       envs directories : /home/user/.conda/envs
                          /mambaforge/envs
               platform : linux-64
             user-agent : conda/4.10.3 requests/2.26.0 CPython/3.9.7 Linux/5.11.0-41-generic ubuntu/20.04.1 glibc/2.31
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

A work around is for the user to remove the .json files.

rm /mambaforge/pkgs/cache/*.json

I have the same problem when I use conda on Windows. Error message is error libmamba Could not open lockfile 'D:\ProgramData\miniconda3\pkgs\cache\cache.lock'. The problem is current user do not have right to write ‘D:\ProgramData\miniconda3\pkgs\cache\cache.lock’`. When I open adaconda power shell as a administer, the problem goes away.

I guess at least for some people that might be identical to #1123 and thus potentially addressed by #2141 .

ie mtime update to a given time denied (unless owned by effective uid), while special calling case of utime or utimensat allows updating timestamp to the current time as long as the file is writable.

I just started seeing this issue out of nowhere as part of my Azure pipeline. During conda mambabuild ..., I see:

RuntimeError: Operation not permitted: '/usr/share/miniconda/pkgs/cache/2ce54b42.json'

I presume something must have changed about permissions in this directory in the Azure docker image for Ubuntu. Similar CI with conda build works as expected.

Just saw this here as well: https://github.com/pandas-dev/pandas/pull/45902

While working on a script to duplicate the issue, I found a fix for my environment. Our install was shared by many users and they had write access to the pkgs & cache folders. Removing this access and setting the gid back to root resolved the issue and now shared pkg updates must be done via sudo. This forces user pkgs & envs into their home/personal environment, but that is ok for now.