hypermodern-python: nox session doesn't find pytest installed by poetry

I’m following along the Hypermodern Python series and it seems in Chapter 2, Test automation with Nox there is a conflict between nox and poetry. Specifically when I use the example noxfile.py:

import nox

@nox.session(python=["3.8", "3.7"])
def tests(session):
    session.run("poetry", "install", external=True)
    session.run("pytest", "--cov")

I get the following error:

nox > Running session tests-3.8
nox > Creating virtual environment (virtualenv) using python3.8 in .nox/tests-3-8
nox > poetry install
Installing dependencies from lock file

No dependencies to install or update

  - Installing testpkg (0.1.0)
nox > Program pytest not found.
nox > Session tests-3.8 failed.

(and the same for tests-3.7)

It seems that installing pytest via poetry as a dev dependency doesn’t put it on the path. In the previous section of Chapter 2, pytest was invoked through poetry and this does work in the nox session as well:

session.run('poetry', 'run', 'pytest', '--cov')

Is this an issue with my setup or is it expected that pytest can’t be run as a command when installed by poetry?

About this issue

Original URL
State: open
Created 4 years ago
Reactions: 1
Comments: 21 (7 by maintainers)

Most upvoted comments

For people arriving here in 2022, I’ve found that I can get it to work if I execute nox within a poetry shell.

oneextrafact on Dec 14, 2022

as per @oneextrafact, the following works

poetry run nox

callmephilip on Oct 27, 2023

Your question helped me find the problem. The nox version was 2019.5.30 which did not match what my other computer was using. I ended up actually reinstalling my Ubuntu instance (running as WSL on Windows 10) and have been worked through the same initialization steps again. I realized that the installation of nox didn’t quite work as expected. Running “pip install --user --upgrade nox” did complete the install, but then the nox keyword would return

Command 'nox' not found, but can be installed with:
sudo apt install nox

So I realized I had installed via sudo which installed the 2019.5.30 version. The underlying issue is that using the pip method to install nox did not add it to PATH, so I just needed to add

export PATH="$HOME/.local/bin:$PATH"

to my .bashrc file and reload the shell. Now it seems to work as expected.

joezilla86 on Oct 18, 2020

I created a corresponding issue.

Dominik1123 on Jun 16, 2020

I just figured it out (though I don’t completely understand why that behavior occurred). Apparently there was a conflict with an active Miniconda environment which was automatically activated through .bashrc (its basically a blank environment). So every time I executed the nox command there was also the Miniconda environment activate. This apparently caused Poetry to create its own venv. After deactivating the Miniconda venv Poetry reuses the nox-venv. Is it because the Miniconda venv shadowed the Nox venv and Poetry realized this and thus created its own venv?

Dominik1123 on Jun 15, 2020

I have been able to reproduce this. Looks like Poetry silently falls back to creating its own virtual environment if it cannot use the virtual environment it’s running in. Can you try removing both the Nox and Poetry environments for your project?

cd path/to/hypermodern-python
rm -rf .nox
poetry env remove 3.8

Specifically, I did the following to reproduce the issue:

git clone https://github.com/cjolowicz/hypermodern-python.git
cd hypermodern-python/
git switch chapter02
nox -s tests -p 3.8
docker run --rm -ti -v $(pwd):/src -w /src python:3.8.3 bash
# in the container:
pip install nox==2020.5.24
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python
source $HOME/.poetry/env
nox -s tests -p 3.8 -r

The output shows that Poetry creates its own virtual environment instead of installing into the existing Nox environment:

nox > Running session tests-3.8
nox > Re-using existing virtual environment at .nox/tests-3-8.
nox > poetry install --no-dev
Creating virtualenv hypermodern-python-VsnhxLU2-py3.8 in /root/.cache/pypoetry/virtualenvs

In this case, the environment is unusable for Poetry because it was created outside of the container. The shebangs in the entrypoint scripts point to a Python interpreter located outside of the container.

nox > pytest --cov -m not e2e
nox > Session tests-3.8 raised exception FileNotFoundError(2, 'No such file or directory')
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/nox/sessions.py", line 462, in execute
    self.func(session)
  File "/usr/local/lib/python3.8/site-packages/nox/_decorators.py", line 53, in __call__
    return self.func(*args, **kwargs)
  File "/src/noxfile.py", line 8, in tests
    session.run("pytest", *args)
  File "/usr/local/lib/python3.8/site-packages/nox/sessions.py", line 226, in run
    return self._run(*args, env=env, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/nox/sessions.py", line 254, in _run
    return nox.command.run(args, env=env, path=self.bin, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/nox/command.py", line 109, in run
    return_code, output = popen(
  File "/usr/local/lib/python3.8/site-packages/nox/popen.py", line 35, in popen
    proc = subprocess.Popen(args, env=env, stdout=stdout, stderr=stderr)
  File "/usr/local/lib/python3.8/subprocess.py", line 854, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/local/lib/python3.8/subprocess.py", line 1702, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/src/.nox/tests-3-8/bin/pytest'
nox > Session tests-3.8 failed.

# head /src/.nox/tests-3-8/bin/pytest
#!/private/tmp/hypermodern-python/.nox/tests-3-8/bin/python
# -*- coding: utf-8 -*-
import re
import sys
from pytest import main
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

cjolowicz on Jun 10, 2020