copier: `pip install copier` fails on windows

Describe the problem Running pip install copier with python 3.10 fails on Windows when the Microsoft C++ build tools package is not installed. I would prefer not install the heavy (7GB) c++ compiler installed just to use copier.

This seems to be caused by the iteration_utilities package lacking a python 3.10 wheel file in pypi and it doesn’t look like they are planning to generate wheel files for python 3.10 any time soon.

To Reproduce

Run pip install copier on Windows with python 3.10 and without visual c++ compiler installed

Logs

Running pip install copier

PS C:\Users\wilson> pip install copier
Collecting copier
  Using cached copier-7.0.1-py3-none-any.whl (34 kB)
Collecting iteration_utilities>=0.11.0
  Using cached iteration_utilities-0.11.0.tar.gz (124 kB)
  Preparing metadata (setup.py) ... done
Collecting pygments>=2.7.1
  Downloading Pygments-2.13.0-py3-none-any.whl (1.1 MB)
     ---------------------------------------- 1.1/1.1 MB 6.3 MB/s eta 0:00:00
Requirement already satisfied: plumbum>=1.6.9 in c:\users\wilson\appdata\local\programs\python\python310\lib\site-packages (from copier) (1.8.0)
Collecting pyyaml-include>=1.2
  Downloading pyyaml_include-1.3-py3-none-any.whl (20 kB)
Collecting questionary>=1.8.1
  Downloading questionary-1.10.0-py3-none-any.whl (31 kB)
Collecting jinja2-ansible-filters>=1.3.1
  Downloading jinja2_ansible_filters-1.3.2-py3-none-any.whl (18 kB)
Requirement already satisfied: packaging>=21.0 in c:\users\wilson\appdata\local\programs\python\python310\lib\site-packages (from copier) (21.3)
Collecting jinja2>=3.1.1
  Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB)
     ---------------------------------------- 133.1/133.1 kB 4.0 MB/s eta 0:00:00
Collecting pydantic>=1.10.2
  Downloading pydantic-1.10.2-cp310-cp310-win_amd64.whl (2.1 MB)
     ---------------------------------------- 2.1/2.1 MB 6.3 MB/s eta 0:00:00
Collecting dunamai>=1.7.0
  Downloading dunamai-1.15.0-py3-none-any.whl (23 kB)
Collecting pyyaml>=5.3.1
  Using cached PyYAML-6.0-cp310-cp310-win_amd64.whl (151 kB)
Requirement already satisfied: colorama>=0.4.3 in c:\users\wilson\appdata\local\programs\python\python310\lib\site-packages (from copier) (0.4.5)
Requirement already satisfied: pathspec>=0.9.0 in c:\users\wilson\appdata\local\programs\python\python310\lib\site-packages (from copier) (0.10.2)
Collecting MarkupSafe>=2.0
  Downloading MarkupSafe-2.1.1-cp310-cp310-win_amd64.whl (17 kB)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in c:\users\wilson\appdata\local\programs\python\python310\lib\site-packages (from packaging>=21.0->copier) (3.0.9)
Requirement already satisfied: pywin32 in c:\users\wilson\appdata\local\programs\python\python310\lib\site-packages (from plumbum>=1.6.9->copier) (305)
Requirement already satisfied: typing-extensions>=4.1.0 in c:\users\wilson\appdata\local\programs\python\python310\lib\site-packages (from pydantic>=1.10.2->copier) (4.4.0)
Collecting prompt_toolkit<4.0,>=2.0
  Downloading prompt_toolkit-3.0.33-py3-none-any.whl (383 kB)
     ---------------------------------------- 383.6/383.6 kB 5.9 MB/s eta 0:00:00
Collecting wcwidth
  Downloading wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Building wheels for collected packages: iteration_utilities
  Building wheel for iteration_utilities (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [15 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-cpython-310
      creating build\lib.win-amd64-cpython-310\iteration_utilities
      copying src\iteration_utilities\_additional_recipes.py -> build\lib.win-amd64-cpython-310\iteration_utilities
      copying src\iteration_utilities\_classes.py -> build\lib.win-amd64-cpython-310\iteration_utilities
      copying src\iteration_utilities\_convenience.py -> build\lib.win-amd64-cpython-310\iteration_utilities
      copying src\iteration_utilities\_recipes.py -> build\lib.win-amd64-cpython-310\iteration_utilities
      copying src\iteration_utilities\_utils.py -> build\lib.win-amd64-cpython-310\iteration_utilities
      copying src\iteration_utilities\__init__.py -> build\lib.win-amd64-cpython-310\iteration_utilities
      running build_ext
      building 'iteration_utilities._iteration_utilities' extension
      error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for iteration_utilities
  Running setup.py clean for iteration_utilities
Failed to build iteration_utilities
Installing collected packages: wcwidth, pyyaml, pygments, pydantic, prompt_toolkit, MarkupSafe, iteration_utilities, questionary, pyyaml-include, jinja2, dunamai, jinja2-ansible-filters, copier
  Running setup.py install for iteration_utilities ... error
  error: subprocess-exited-with-error

  × Running setup.py install for iteration_utilities did not run successfully.
  │ exit code: 1
  ╰─> [17 lines of output]
      running install
      C:\Users\wilson\AppData\Local\Programs\Python\Python310\lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-cpython-310
      creating build\lib.win-amd64-cpython-310\iteration_utilities
      copying src\iteration_utilities\_additional_recipes.py -> build\lib.win-amd64-cpython-310\iteration_utilities
      copying src\iteration_utilities\_classes.py -> build\lib.win-amd64-cpython-310\iteration_utilities
      copying src\iteration_utilities\_convenience.py -> build\lib.win-amd64-cpython-310\iteration_utilities
      copying src\iteration_utilities\_recipes.py -> build\lib.win-amd64-cpython-310\iteration_utilities
      copying src\iteration_utilities\_utils.py -> build\lib.win-amd64-cpython-310\iteration_utilities
      copying src\iteration_utilities\__init__.py -> build\lib.win-amd64-cpython-310\iteration_utilities
      running build_ext
      building 'iteration_utilities._iteration_utilities' extension
      error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> iteration_utilities

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

[notice] A new release of pip available: 22.2.1 -> 22.3.1
[notice] To update, run: python.exe -m pip install --upgrade pip

Likewise when running poetry install as part of the copier contribution dev guidelines


PS D:\repositories\MakerX\copier> poetry config virtualenvs.in-project true --local
PS D:\repositories\MakerX\copier> poetry install --with docs
Creating virtualenv copier in D:\repositories\MakerX\copier\.venv
Installing dependencies from lock file

Package operations: 83 installs, 1 update, 0 removals

  • Installing six (1.16.0)
  • Installing colorama (0.4.6)
  • Installing markupsafe (2.1.1)
  • Installing pyparsing (3.0.9)
  • Installing python-dateutil (2.8.2)
  • Installing pyyaml (6.0)
  • Installing zipp (3.8.1)
  • Installing attrs (22.1.0)
  • Installing click (8.1.3)
  • Installing ghp-import (2.1.0)
  • Installing jinja2 (3.1.2)
  • Installing mergedeep (1.3.4)
  • Installing markdown (3.3.5)
  • Installing packaging (21.3)
  • Installing pluggy (1.0.0)
  • Installing importlib-metadata (4.2.0)
  • Installing pyyaml-env-tag (0.1)
  • Installing py (1.11.0)
  • Installing iniconfig (1.1.1)
Connection pool is full, discarding connection: pypi.org. Connection pool size: 10
Connection pool is full, discarding connection: pypi.org. Connection pool size: 10
  • Installing tomli (2.0.1)
  • Installing watchdog (2.1.9)
  • Installing wcwidth (0.2.5)
  • Installing distlib (0.3.6)
  • Installing filelock (3.8.0)
  • Installing mkdocs (1.2.4)
  • Installing mccabe (0.7.0)
  • Installing griffe (0.22.1)
  • Installing pyflakes (2.5.0)
  • Installing pytest (7.1.3)
  • Installing pycodestyle (2.9.1)
  • Installing platformdirs (2.5.2)
  • Installing mkdocstrings (0.19.0)
  • Installing prompt-toolkit (3.0.31)
  • Updating setuptools (65.5.1 -> 65.3.0)
Connection pool is full, discarding connection: pypi.org. Connection pool size: 10
Connection pool is full, discarding connection: pypi.org. Connection pool size: 10
  • Installing argcomplete (1.12.3)
  • Installing astor (0.8.1)
  • Installing cfgv (3.3.1)
  • Installing decli (0.5.2)
  • Installing coverage (6.4.4)
  • Installing mkdocs-autorefs (0.4.1)
  • Installing mkdocstrings-python (0.7.1)
  • Installing identify (2.5.5)
  • Installing execnet (1.9.0)
  • Installing charset-normalizer (2.1.1)
  • Installing flake8 (5.0.4)
  • Installing mkdocs-material-extensions (1.0.3)
Connection pool is full, discarding connection: pypi.org. Connection pool size: 10
  • Installing mypy-extensions (0.4.3)
  • Installing nodeenv (1.7.0)
  • Installing pastel (0.2.1)
  • Installing pathspec (0.10.1)
  • Installing ptyprocess (0.7.0)
  • Installing pygments (2.13.0)
  • Installing pymdown-extensions (9.5)
  • Installing pytest-forked (1.4.0)
  • Installing pywin32 (304)
  • Installing questionary (1.10.0)
  • Installing termcolor (1.1.0)
  • Installing toml (0.10.2)
  • Installing tomlkit (0.11.4)
  • Installing typing-extensions (4.4.0)
  • Installing virtualenv (20.16.2)
  • Installing autoflake (1.6.1)
  • Installing black (22.8.0)
  • Installing flake8-bugbear (22.9.23)
  • Installing dunamai (1.14.1)
  • Installing commitizen (2.35.0)
  • Installing iteration-utilities (0.11.0)
  • Installing jinja2-ansible-filters (1.3.2)
  • Installing flake8-simplify (0.19.3)
  • Installing mkdocs-material (8.2.6)
  • Installing flake8-comprehensions (3.10.0)
  • Installing isort (5.10.1)
  • Installing flake8-debugger (4.1.2)
Connection pool is full, discarding connection: pypi.org. Connection pool size: 10
Connection pool is full, discarding connection: pypi.org. Connection pool size: 10
  • Installing mypy (0.981)
  • Installing pexpect (4.8.0)
  • Installing plumbum (1.8.0)
  • Installing poethepoet (0.16.2)
  • Installing pre-commit (2.20.0)
  • Installing pydantic (1.10.2)
  • Installing pytest-cov (4.0.0)
  • Installing pytest-xdist (2.5.0)
  • Installing pyyaml-include (1.3)
  • Installing types-backports (0.1.3)
  • Installing types-pyyaml (6.0.12)

  CalledProcessError

  Command 'D:\repositories\MakerX\copier\.venv\Scripts\python.exe -m pip install --use-pep517 --disable-pip-version-check --prefix D:\repositories\MakerX\copier\.venv --no-deps C:\Users\wilson\AppData\Local\pypoetry\Cache\artifacts\13\fe\58\dd3a5413393eab8ca2c5b884ff40af30da952dbdc4598df1d7b6cf61b0\iteration_utilities-0.11.0.tar.gz' returned non-zero exit status 1.

  at ~\AppData\Local\Programs\Python\Python310\lib\subprocess.py:524 in run
       520│             # We don't call process.wait() as .__exit__ does that for us.
       521│             raise
       522│         retcode = process.poll()
       523│         if check and retcode:
    →  524│             raise CalledProcessError(retcode, process.args,
       525│                                      output=stdout, stderr=stderr)
       526│     return CompletedProcess(process.args, retcode, stdout, stderr)
       527│
       528│

The following error occurred when trying to handle this error:


  EnvCommandError

  Command D:\repositories\MakerX\copier\.venv\Scripts\python.exe -m pip install --use-pep517 --disable-pip-version-check --prefix D:\repositories\MakerX\copier\.venv --no-deps C:\Users\wilson\AppData\Local\pypoetry\Cache\artifacts\13\fe\58\dd3a5413393eab8ca2c5b884ff40af30da952dbdc4598df1d7b6cf61b0\iteration_utilities-0.11.0.tar.gz errored with the following return code 1, and output:
  Processing c:\users\wilson\appdata\local\pypoetry\cache\artifacts\13\fe\58\dd3a5413393eab8ca2c5b884ff40af30da952dbdc4598df1d7b6cf61b0\iteration_utilities-0.11.0.tar.gz
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
    Preparing metadata (pyproject.toml): started
    Preparing metadata (pyproject.toml): finished with status 'done'
  Building wheels for collected packages: iteration-utilities
    Building wheel for iteration-utilities (pyproject.toml): started
    Building wheel for iteration-utilities (pyproject.toml): finished with status 'error'
    error: subprocess-exited-with-error

    Building wheel for iteration-utilities (pyproject.toml) did not run successfully.
    exit code: 1

    [15 lines of output]
    running bdist_wheel
    running build
    running build_py
    creating build
    creating build\lib.win-amd64-cpython-310
    creating build\lib.win-amd64-cpython-310\iteration_utilities
    copying src\iteration_utilities\_additional_recipes.py -> build\lib.win-amd64-cpython-310\iteration_utilities
    copying src\iteration_utilities\_classes.py -> build\lib.win-amd64-cpython-310\iteration_utilities
    copying src\iteration_utilities\_convenience.py -> build\lib.win-amd64-cpython-310\iteration_utilities
    copying src\iteration_utilities\_recipes.py -> build\lib.win-amd64-cpython-310\iteration_utilities
    copying src\iteration_utilities\_utils.py -> build\lib.win-amd64-cpython-310\iteration_utilities
    copying src\iteration_utilities\__init__.py -> build\lib.win-amd64-cpython-310\iteration_utilities
    running build_ext
    building 'iteration_utilities._iteration_utilities' extension
    error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
    [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
    ERROR: Failed building wheel for iteration-utilities
  Failed to build iteration-utilities
  ERROR: Could not build wheels for iteration-utilities, which is required to install pyproject.toml-based projects


  at ~\AppData\Local\Programs\Python\Python310\lib\site-packages\poetry\utils\env.py:1476 in _run
      1472│                 output = subprocess.check_output(
      1473│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1474│                 )
      1475│         except CalledProcessError as e:
    → 1476│             raise EnvCommandError(e, input=input_)
      1477│
      1478│         return decode(output)
      1479│
      1480│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:

The following error occurred when trying to handle this error:


  PoetryException

  Failed to install C:/Users/wilson/AppData/Local/pypoetry/Cache/artifacts/13/fe/58/dd3a5413393eab8ca2c5b884ff40af30da952dbdc4598df1d7b6cf61b0/iteration_utilities-0.11.0.tar.gz

  at ~\AppData\Local\Programs\Python\Python310\lib\site-packages\poetry\utils\pip.py:51 in pip_install
       47│
       48│     try:
       49│         return environment.run_pip(*args)
       50│     except EnvCommandError as e:
    →  51│         raise PoetryException(f"Failed to install {path.as_posix()}") from e
       52│

Warning: The file chosen for install of markdown 3.3.5 (Markdown-3.3.5-py3-none-any.whl) is yanked. Reason for being yanked: Invalid dependency specified. Fixed in 3.3.6.

Expected behavior pip install copier should install copier without error

Environment

OS: Windows 10
Copier version: 7.0.1
Python version: 3.10
Installation method: pip+pypi

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 16 (12 by maintainers)

Most upvoted comments

I don’t have a specific plan, but there are some pending things I’d like to be in the next release. Usually you can install from master, but if you need wheels or stuff like that, I can push an alpha tag.

yajo on Dec 20, 2022

Hi @pawamoy, thanks for the suggestion. I agree that asking the iteration-utilities team to build wheels for python 3.10 would be the best solution. However the existing issue on their bugtracker has been open for 6+ months so it seems it may not get resolved. I also note the project hasn’t had any commits for 2+ years.

I thought it might be better to solve the issue by removing the dependency on iteration-utilities. Which is done in #890 .

wilsonwaters on Dec 6, 2022

Yep sorry, I was waiting to have some time to analyze what changelog is produced. I use commitizen for tags and it has https://github.com/commitizen-tools/commitizen/issues/364.

Well, it won’t be the changelog I’d love to have, but it’s something. When fixed upstream, we’ll have a better one.

Summarizing, there you have: https://github.com/copier-org/copier/releases/tag/v7.1.0a0 (in some minutes there should be a wheel pushed on pypi)

yajo on Dec 29, 2022

Definitely. Thanks for engaging with us on this 😃

We’ll get a pull request over early in the week.

robdmoore on Dec 17, 2022

Hi @yajo,

I understand and agree with the general principle that you’ve outlined of minimising code, especially for functionality that’s not part of the core of copier. Every bit of code you add of course has a cost. However, in this case I’d ask that you reconsider given:

The package in question is used by one line of code in copier, for use of a single function from a package that has 91 functions (i.e. we are using a trivial part of the library that is being taken as a dependency)
The function in question isn’t terribly complex either and is replaced with a simple and elegant 8 line Python function
This new Python function is “done” - the function does one clear thing, does it well, shouldn’t need to change, and also has full test coverage
The dependency itself (iteration_utilities) is clearly out of date (no commits for 2 years), and while if it was full of pure Python functions that were similarly “done” it wouldn’t be an issue, but in this case the library is full of C code that needs either wheels to be maintained as new Python versions (which isn’t happening) or a source build when you install, which slows down the install time of copier on all systems
On Windows the problem is significantly worse, given the C++ build tools are not on by default, even with standard developer setups it means that you need to download a 7GB install and reboot your computer, just to use copier
We have incorporated copier into a CLI tool that we have created to create an amazing dev experience for a particular technology ecosystem, that includes an init command with templating. Copier is frankly incredible for this use case, the functionality is top class and the experience we are able to provide is phenominal. Thank you!
- Importantly, the goal of the tool we are building, which will be used by thousands of developers, is to make it really easy to quickly get up and running, regardless of what the setup is on your computer or your operating system. It’s not feasible to ask people to install a 7GB install and reboot their machine just to use the CLI tool - they won’t do it.

Given all of that, hopefully you can agree that in this instance, swapping out iteration_utilities is a no brainer option that will significantly improve the copier experience?

robdmoore on Dec 16, 2022