pytest: pytest2.8 invariantly writes to working directory + fails on readonly filesystem

I believe these to be the same issue so I’m only making a single report.

$ virtualenv venv
...
$ . venv/bin/activate
$ pip install pytest
...
$ py.test foo.py
============================= test session starts ==============================
platform linux2 -- Python 2.7.6, pytest-2.8.0, py-1.4.30, pluggy-0.3.1
rootdir: /tmp/foo, inifile: 

===============================  in 0.00 seconds ===============================
ERROR: file not found: foo.py
(venv)asottile@work:/tmp/foo$ ls -al .cache/v/cache/lastfailed 
-rw-rw-r-- 1 asottile asottile 2 Sep 20 17:03 .cache/v/cache/lastfailed

And the error from our CI server (which is running a repo inside docker)

15:06:43 docker run -t -v /nail/scratch/jenkins_prod_slave/workspace/packages-ubuntu-allocate_playground:/work:ro nginx_test_container /bin/bash -c "cd /work/itest && . /venv34/bin/activate && py.test -s test_nginx.py"
15:06:43 ============================= test session starts ==============================
15:06:43 platform linux -- Python 3.4.2, pytest-2.8.0, py-1.4.30, pluggy-0.3.1
15:06:43 rootdir: /work, inifile: 
15:06:43 
collecting 0 items
collecting 5 items
collected 5 items 
15:06:43 
15:06:44 test_nginx.py
...
15:06:51 .Traceback (most recent call last):
15:06:51   File "/venv34/lib/python3.4/site-packages/py/_error.py", line 64, in checked_call
15:06:51     return func(*args, **kwargs)
15:06:51 OSError: [Errno 30] Read-only file system: '/work/.cache/v/cache/lastfailed'
15:06:51 
15:06:51 During handling of the above exception, another exception occurred:
15:06:51 
15:06:51 Traceback (most recent call last):
15:06:51   File "/venv34/bin/py.test", line 11, in <module>
15:06:51     sys.exit(main())
15:06:51   File "/venv34/lib/python3.4/site-packages/_pytest/config.py", line 48, in main
15:06:51     return config.hook.pytest_cmdline_main(config=config)
15:06:51   File "/venv34/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 724, in __call__
15:06:51     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
15:06:51   File "/venv34/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
15:06:51     return self._inner_hookexec(hook, methods, kwargs)
15:06:51   File "/venv34/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
15:06:51     _MultiCall(methods, kwargs, hook.spec_opts).execute()
15:06:51   File "/venv34/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 596, in execute
15:06:51     res = hook_impl.function(*args)
15:06:51   File "/venv34/lib/python3.4/site-packages/_pytest/main.py", line 115, in pytest_cmdline_main
15:06:51     return wrap_session(config, _main)
15:06:51   File "/venv34/lib/python3.4/site-packages/_pytest/main.py", line 110, in wrap_session
15:06:51     exitstatus=session.exitstatus)
15:06:51   File "/venv34/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 724, in __call__
15:06:51     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
15:06:51   File "/venv34/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
15:06:51     return self._inner_hookexec(hook, methods, kwargs)
15:06:51   File "/venv34/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
15:06:51     _MultiCall(methods, kwargs, hook.spec_opts).execute()
15:06:51   File "/venv34/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 595, in execute
15:06:51     return _wrapped_call(hook_impl.function(*args), self.execute)
15:06:51   File "/venv34/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 249, in _wrapped_call
15:06:51     wrap_controller.send(call_outcome)
15:06:51   File "/venv34/lib/python3.4/site-packages/_pytest/terminal.py", line 361, in pytest_sessionfinish
15:06:51     outcome.get_result()
15:06:51   File "/venv34/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 278, in get_result
15:06:51     raise ex[1].with_traceback(ex[2])
15:06:51   File "/venv34/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 264, in __init__
15:06:51     self.result = func()
15:06:51   File "/venv34/lib/python3.4/site-packages/_pytest/vendored_packages/pluggy.py", line 596, in execute
15:06:51     res = hook_impl.function(*args)
15:06:51   File "/venv34/lib/python3.4/site-packages/_pytest/cacheprovider.py", line 140, in pytest_sessionfinish
15:06:51     config.cache.set("cache/lastfailed", self.lastfailed)
15:06:51   File "/venv34/lib/python3.4/site-packages/_pytest/cacheprovider.py", line 73, in set
15:06:51     with path.open("w") as f:
15:06:51   File "/venv34/lib/python3.4/site-packages/py/_path/local.py", line 353, in open
15:06:51     return py.error.checked_call(open, self.strpath, mode)
15:06:51   File "/venv34/lib/python3.4/site-packages/py/_error.py", line 84, in checked_call
15:06:51     raise cls("%s%r" % (func.__name__, args))
15:06:51 py.error.EROFS: [Read-only file system]: open('/work/.cache/v/cache/lastfailed', 'w')
15:06:51 make: *** [itest] Error 1
...

About this issue

Original URL
State: closed
Created 9 years ago
Comments: 24 (20 by maintainers)

Commits related to this issue

fix per https://github.com/pytest-dev/pytest/issues/1029 — committed to alvinwan/TexSoup by alvinwan 7 years ago

Most upvoted comments

@asottile

How do I get pytest to stop writing to my working directory and why did this change happen?

I think the only way to prevent that currently is by passing -p no:cacheprovider in the command line, or by modifying your pytest.ini:

[pytest]
addopts = -p no:cacheprovider

(In fact I will add this to the docs)

This change happened because in 2.8 pytest-cache has been merged into the core. It brings with two important functionalities: re-run last failures or failures first and config.cache object, which lets plugins persist data between test sessions.

One of the drawbacks is that now pytest will always create a .cache directory in the rootdir of the test session, as the cache has to be active in order to --lf do its work. We did not foreseen the problems it could cause like your issue demonstrated, unfortunately (which already has been addressed in #1048, btw).

This will be fixed in 2.8.1 when it comes out (soon), but until then you can disable the cache completely as I noted above.

nicoddemus on Sep 23, 2015

@mgedmin thanks. Just want to mention a workaround: -p no:cacheprovider will disable the cache plugin and the problem should go away.

nicoddemus on Oct 27, 2017

@RonnyPfannschmidt That’s wrongheaded on several counts.

a) A very convenient, and standard, way to configure tools’ cache location is to set $XDG_CACHE_DIR. Many of my project’s fixtures do this for other reasons already. Similarly, the most convenient, and standard, way to set “basetmp” would be $TMPDIR.

b) “something a CI system can easily pick up” is exactly these environment variables. Many systems will already support this, and those that don’t will support injecting environment variables.

c) “non-containered ci” will either support ~/.cache correctly, or set $XDG_CACHE_DIR, because this is necessary for other tools that use the standard. Again, this is already handled in many of my projects because of other tools.

bukzor on Sep 29, 2015

The standard place to put this is ~/.cache. To easily disambiguate caches against various $PWD’s, you can mkdir -p $HOME/.cache/pytest/$PWD. Even better would be to use $XDG_CACHE_HOME with a default value of $HOME/.cache.

bukzor on Sep 23, 2015