hypermodern-python: Dependency errors when using python 3.9 and 3.8

Issue

When using python version 3.9: For me, all nox sessions where install_with_constrains is used with python version 3.9 fail.

For instance running the tests session works fine, till chapter 4. After adding more nox sessions and installing more dependencies I started to see multiple dependencies for different python versions (I am following the tutorial using python 3.9 and 3.8 respectively).

After some research, it seems this problem is related to Duplicate dependency export #1970

Console Output

CLICK ME 

nox > Running session tests-3.9
nox > Re-using existing virtual environment at .nox/tests-3-9.
nox > poetry install --no-dev
Installing dependencies from lock file

Package operations: 2 installs, 0 updates, 11 removals

  • Removing coverage (5.3)
  • Removing iniconfig (1.1.1)
  • Removing packaging (20.4)
  • Removing pluggy (0.13.1)
  • Removing py (1.9.0)
  • Removing pyparsing (2.4.7)
  • Removing pytest (6.1.1)
  • Removing pytest-cov (2.10.1)
  • Removing pytest-mock (3.3.1)
  • Removing six (1.15.0)
  • Removing toml (0.10.1)
  • Installing mypy-extensions (0.4.3)
  • Installing typing-extensions (3.7.4.3)

Installing the current project: hypermodern-bersten (0.1.0)
nox > poetry export --dev --format=requirements.txt --output=/tmp/tmppd82bei6
nox > pip install --constraint=/tmp/tmppd82bei6 coverage[toml] pytest pytest-cov pytest-mock
nox > Command pip install --constraint=/tmp/tmppd82bei6 coverage[toml] pytest pytest-cov pytest-mock failed with exit code 1:
Ignoring atomicwrites: markers 'python_version >= "3.5" and python_full_version < "3.0.0" and sys_platform == "win32" and (python_version >= "3.5" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.5") or sys_platform == "win32" and python_version >= "3.5" and python_full_version >= "3.4.0" and (python_version >= "3.5" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.5")' don't match your environment
Ignoring attrs: markers 'python_version == "3.8" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version == "3.8"' don't match your environment
Ignoring colorama: markers 'python_version >= "3.5" and python_full_version < "3.0.0" and sys_platform == "win32" and (python_version >= "3.5" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.5") and platform_system == "Windows" or sys_platform == "win32" and python_version >= "3.5" and python_full_version >= "3.5.0" and (python_version >= "3.5" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.5") and platform_system == "Windows"' don't match your environment
Ignoring decorator: markers 'python_version == "3.8" and python_full_version < "3.0.0" and python_full_version >= "2.7.0" or python_version == "3.8" and python_full_version >= "3.2.0"' don't match your environment
Ignoring importlab: markers 'python_version == "3.8" and python_full_version >= "2.7.0"' don't match your environment
Ignoring networkx: markers 'python_version == "3.8" and python_full_version >= "2.7.0"' don't match your environment
Ignoring ninja: markers 'python_version == "3.8"' don't match your environment
Ignoring pytype: markers 'python_version == "3.8"' don't match your environment
Ignoring pyyaml: markers 'python_version == "3.8" and python_full_version < "3.0.0" or python_version == "3.8" and python_full_version >= "3.5.0"' don't match your environment
Ignoring six: markers 'python_version == "3.8" and python_full_version < "3.0.0" and python_full_version >= "2.7.0" or python_version == "3.8" and python_full_version >= "3.4.0"' don't match your environment
Ignoring typed-ast: markers 'python_version == "3.8"' don't match your environment
Collecting coverage==5.3
  Using cached coverage-5.3-cp39-cp39-manylinux1_x86_64.whl (228 kB)
Collecting pytest-cov==2.10.1
  Using cached pytest_cov-2.10.1-py2.py3-none-any.whl (19 kB)
Collecting pytest-mock==3.3.1
  Using cached pytest_mock-3.3.1-py3-none-any.whl (11 kB)
Collecting pytest==6.1.1
  Using cached pytest-6.1.1-py3-none-any.whl (272 kB)
Collecting toml==0.10.1
  Using cached toml-0.10.1-py2.py3-none-any.whl (19 kB)
Requirement already satisfied: attrs>=17.4.0 in ./.nox/tests-3-9/lib/python3.9/site-packages (from pytest==6.1.1->-c /tmp/tmppd82bei6 (line 176)) (20.2.0)
Collecting pluggy==0.13.1
  Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting iniconfig==1.1.1
  Using cached iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Collecting packaging==20.4
  Using cached packaging-20.4-py2.py3-none-any.whl (37 kB)
Collecting py==1.9.0
  Using cached py-1.9.0-py2.py3-none-any.whl (99 kB)
Collecting pyparsing==2.4.7
  Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting six
ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
    six from https://files.pythonhosted.org/packages/ee/ff/48bde5c0f013094d729fe4b0316ba2a24774b3ff1c52d924a8a4cb04078a/six-1.15.0-py2.py3-none-any.whl#sha256=8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced (from packaging==20.4->-c /tmp/tmppd82bei6 (line 146))
nox > Session tests-3.9 failed.

About this issue

  • Original URL
  • State: open
  • Created 4 years ago
  • Comments: 23 (5 by maintainers)

Commits related to this issue

Most upvoted comments

For everyone not using the Cookiecutter and/or nox-poetry, I would recommend passing --without-hashes to poetry export, and using pip’s new resolver.

+2
cjolowicz on Jan 29, 2021

This seems to continue to persist in 3.10.

$ nox -rs mypy
nox > Running session mypy
nox > Creating virtual environment (virtualenv) using python.BAT in .nox\mypy
nox > poetry export --dev --format=requirements.txt '--output=C:\Users\RITCHI~1.JOS\AppData\Local\Temp\tmpxcsboakw'
nox > python -m pip install '--constraint=C:\Users\RITCHI~1.JOS\AppData\Local\Temp\tmpxcsboakw' mypy
nox > Command python -m pip install '--constraint=C:\Users\RITCHI~1.JOS\AppData\Local\Temp\tmpxcsboakw' mypy failed with exit code 1:
Collecting mypy
ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
    mypy from https://files.pythonhosted.org/packages/37/bf/5729b25e84b0048f4ca249a10f5d5c1cc9210b40a7496435b4b872b874b6/mypy-0.931-cp310-cp310-win_amd64.whl#sha256=7b3f6f557ba4afc7f2ce6d3215d5db279bcf120b3cfd0add20a5d4f4abdae5bc
nox > Session mypy failed.
+1
flickerfly on Jan 11, 2022

hi @staticdev

@cjolowicz any chance we get a new release of cookiecutter with these updated versions?

2021.1.29 was just released.

+1
cjolowicz on Jan 29, 2021

The cause definitely seems to be the changes to pip with the new resolver becoming the default in version 20.3. Looks like the way constraints file work has changed in this version. https://github.com/pypa/pip/issues/9020 and https://pip.pypa.io/en/latest/user_guide/#watch-out-for cleared it up for me.

It seems that the only solution, other than not upgrading pip, is to add “–without-hashes” to the “poetry export” command in install_with_constraints. I couldn’t find anything else, anyway.

+1
Harrison88 on Dec 15, 2020

For me, doing the environment variable approach recommended above and then also pinning the Pip version itself together worked out fine.

One difference between your build and mine is that you’re using the latest version of Nox, whereas we’ve pinned the previous version of it (2020.5.24). Nox has a virtualenv as a dependency, so I wonder if that might somehow be getting in the way.

Also, we’re using poetry==1.0.9 as a pinned install rather than the latest version. Not sure where the issue is coming from for you but my guess is probably that it’s coming from deeper in the dependency tree with the version numbers involved of everything else.

+1
wanderrful on Dec 14, 2020

@wanderrful That didn’t work for me =/ https://github.com/DontShaveTheYak/cloud-radar/actions/runs/420486474

Unless you mean pinning pip with poetry or some other way? Pinning virtualenv worked. I did not try the environment variable approach.

I ended up just using --without-hashes as pinning the version was not a long-term fix since it’s not a bug but the behavior going forward.

+1
shadycuz on Dec 14, 2020

Glad you found a workaround. I would suspect that this is triggered by changes to constraints file handling in pip’s new resolver. Pip is bundled with virtualenv. You could then also “solve” this by pinning pip to a version before the new resolver became the default, setting the environment variable VIRTUALENV_PIP to 20.2.4

+1
cjolowicz on Dec 11, 2020

@wanderrful

Here is what I am currently using:

def install_with_constraints(session: Session, *args: str, **kwargs: Any) -> None:
    """Install packages constrained by Poetry's lock file.

    This function is a wrapper for nox.sessions.Session.install. It
    invokes pip to install packages inside of the session's virtualenv.
    Additionally, pip is passed a constraints file generated from
    Poetry's lock file, to ensure that the packages are pinned to the
    versions specified in poetry.lock. This allows you to manage the
    packages as Poetry development dependencies.

    Arguments:
        session: The Session object.
        args: Command-line arguments for pip.
        kwargs: Additional keyword arguments for Session.install.
    """
    with tempfile.NamedTemporaryFile() as requirements:
        session.run(
            "poetry",
            "export",
            "--dev",
            "--format=requirements.txt",
            "--without-hashes",
            f"--output={requirements.name}",
            external=True,
        )
        session.install(f"--constraint={requirements.name}", *args, **kwargs)

Then using it within nox sessions as:

@nox.session(python=_versions)
def lint(session: Session) -> None:
    """Run the code linters."""
    args = session.posargs or locations
    install_with_constraints(
        session,
        "darglint",
        "flake8",
        "flake8-annotations",
        "flake8-black",
        "flake8-docstrings",
        "flake8-isort",
        "flake8-rst-docstrings",
        "flake8_sphinx_links",
    )
    session.run("flake8", *args)

I hope that helps.

+1
ToddG on Dec 11, 2020

https://github.com/cjolowicz/hypermodern-python/blob/400386bd73109fa2163ddfd05c05b6dc7d316afc/noxfile.py#L30-L39

I fixed this problem by adding --without-hashes parameter to the install_with_constraints function in noxfile.

+1
mjt91 on Nov 3, 2020
Read more comments on GitHub
← ha-myenergi: Zappi energy used/green energy used not set in HA
hypermodern-python: nox session doesn't find pytest installed by poetry →