pip: Deprecate call to `setup.py install` when building a wheel failed for source distributions without pyproject.toml

I was surprised to see so many link-backs for a simple deprecation message, and even downvotes. And then I realised each of the linked issues simply contains an error message of someone failing to build an arbitrary package from source with pip install (which would contain a link to this issue), and people don’t realise the deprecation message has nothing to do with it.

Maybe we should have only show the message if the setup.py install command succeeds instead of unconditionally. This would save us a lot of confused users blaming this issue for their problems.

This discussion is spiralling faster than I can keep up. Since I’m trying to be on vacation, I’m gonna step away from this after this response.

I don’t understand how you could realize that for public-pypi and not consider private-pypi at all in your transition plans.

I am concerned that you guys are cagey and don’t want to go on the record about whether pip supports private registries.

I’m not being cagey – I have limited time and there’s way too much for me to respond to. And, as I’ve already stated, posting a wall of questions isn’t particularly conducive to constructive discussion – which I imagine contributed to you never getting a response to this from anyone else.

As far the question goes, pip does support private package indexes and I can say that we do make changes and do consider their usage a lot when designing things and planning migrations.

However, that doesn’t mean that private package indexes can do whatever though. Nor does it mean that pip needs to support that “whatever” indefinitely. Being able to use the latest pip which contains potentially-breaking changes with non-compliant private package indexes isn’t a reasonable expectation.

If organisations/commercial providers can’t move at the speed that we move at, that’s fine – it’s reasonable that they stick to older versions of open source software in those situations and opt out of “free” support from the maintainers of that open source software.

don’t have pip install . telling me to remove it and pip install -e . telling me to add it back!

Got an example of this? If this is happening, that’s a bug.

folks here are basically saying “the tools contradict each other, we know that, we know it’s bad for users, and we think users should accept this status quo anyway”.

Here’s my attempt to summarize the status quo…

We’re going to be bringing tools into a consistent and saner model, and things should be less bad on the other side for everyone. There’s going to be a migration cost and pip is the latest piece to start warning people about the upcoming migration. Python packaging tools are all going to be migrating in the future (or already have) but their timelines may not be as aligned as a full-time funded team of engineers and project managers would be able to provide to end users.

@drtoche the important part is WARNING: Built wheel for my_package_name is invalid: Metadata 1.2 mandates PEP 440 version, but 'dev' is not. Your package has an version number (dev) which is invalid, therefore pip rejects the wheel that was built from it, and falls back to the legacy setup.py install method which is less strict

Setting a valid version number in your project should fix both the warning and the deprecation.

@sylwesterdigital thanks. The relevant error is

    C:\Users\Public\Documents\Wondershare\CreatorTemp\pip-install-enu1m14b\pysqlcipher3\src\python3\connection.h(33): fatal error C1083: Cannot open include file: 'sqlcipher/sqlite3.h': No such file or directory
    error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.27.29110\\bin\\HostX86\\x64\\cl.exe' failed with exit status 2

So the sqlcipher dependency is missing.

We can also see that pysqlcipher3 is not installed despite setup.py install reporting success.

So I suspect there is a bug in pysqlcipher3’s setup.py that causes it to report success while it is actually failing. I’d suggest reporting the issue with that project, as there is nothing we can do on pip side.

One way to make pip “stricter” is --use-pep517. Stricter in quotes, because this is not exactly the same as disabling setup.py install; the option basically makes pip always pretend there is a pyproject.toml for each package (instead of using the globally installed setuptools), but that should introduce no functional differences for most packages. And since projects with pyproject.toml are never installed with setup.py install, the option kind of achieves what you want.

Thanks for clearly providing all the relevant information here @djhenderson. I think the issue tracker for Pillow (where you’ve filed an issue) is the correct place to get help with your issue. 😃

@osmalpkoras the build error for gensim was likely printed and you should raise it with that project.

Seek guidance, python3.10.2 pyzmq19.0.2 installation failure, where is the problem?

Building wheel for pyzmq (setup.py) … error error: subprocess-exited-with-error × python setup.py bdist_wheel did not run successfully. │ exit code: 1 ╰─> [248 lines of output] C:\Users\无双破军\AppData\Local\Temp\pip-req-build-5npw69cc\setup.py:1140: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. if V(Cython.version) < V(min_cython_version): C:\Users\无双破军\AppData\Local\Temp\pip-req-build-5npw69cc\setup.py:1148: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. Warning: Couldn’t find an acceptable libzmq on the system. Warning: Failed to get cc._vcruntime via private API, not bundling CRT AttributeError: ‘MSVCCompiler’ object has no attribute ‘_vcruntime_redist’ [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for pyzmq Running setup.py clean for pyzmq Failed to build pyzmq Installing collected packages: pyzmq Running setup.py install for pyzmq … done DEPRECATION: pyzmq was installed using the legacy ‘setup.py install’ method, because a wheel could not be built for it. A possible replacement is to fix the wheel build issue reported above.

is the legacy setup.py going to get discontinued anytime soon

Yes. I’m not sure without checking what the timescale is, but we will be removing this fallback.

Also, a follow-up question, will the above versioning cause build failures if a wheel doesn’t get built

I’m not sure what you mean by this, what sort of build are you thinking of apart from building a wheel? All forms of direct build via setup.py have now been deprecated (by setuptools, not by pip) so at some point you’ll have to move to a PEP 440 compliant version. And even if you don’t get problems with build failures, you’ll start hitting issues with other tools which only support PEP 440 style versions.

because i really want to use custom release names

Sorry, the standards don’t support them, so you’ll increasingly find that standard tools will reject them or fail.

I don’t have the resources to fix the symlink problem. I am commenting here because the limitation imposed by deprecating “pip install” of an sdist running setup.py without creating a wheel creates a problem right now because there is no other accepted symlink solution (in answer to the question above by @sbidoul : “it would be interesting to understand why setup.py bdist_wheel would not work for you, and what would block you from moving to PEP 517”)

Many thanks for offering the help. The problem has been solved as I changed to install a different version of the package. Thanks

Difang Deng

---

From: Stéphane Bidoul notifications@github.com Sent: Saturday, January 9, 2021 12:07:18 PM To: pypa/pip pip@noreply.github.com Cc: ddfcrystal difangdeng@outlook.com; Mention mention@noreply.github.com Subject: Re: [pypa/pip] Deprecate call to ‘setup.py install’ when building a wheel failed for source distributions without pyproject.toml (#8368)

@ddfcrystalhttps://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fddfcrystal&data=04|01||a92c61de73c74a5afc6808d8b4971ce2|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637457908396685117|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=XzkAPqJ5cSwGoxR%2Bh%2BVcM97vIpL%2FNScVZY1Ud5e4YF8%3D&reserved=0 the issue most probably lies in the setup.py of the package you try to install. Feel free however to post your installation log here and I’ll take a look.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpypa%2Fpip%2Fissues%2F8368%23issuecomment-757139971&data=04|01||a92c61de73c74a5afc6808d8b4971ce2|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637457908396685117|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=ZdeRtWYjaZomhISjSl1H3CmFZytwB1eeDsRlniMWtfU%3D&reserved=0, or unsubscribehttps://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FARSTVDHKJEKQ6P5T7YFRYY3SZBBHNANCNFSM4NPEEELQ&data=04|01||a92c61de73c74a5afc6808d8b4971ce2|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637457908396685117|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=BllJa2IGRkj1qZyJ2m4Xo%2F%2BrI8IVEnHnacTBEJiGS0o%3D&reserved=0.

I’m rather confused by this. When I install my library (mindsdb, for which I explicitly provide no wheel since it requires user-specific steps in setup.py for it to install properly) I get:

DEPRECATION: mindsdb was installed using the legacy 'setup.py install' method, because a wheel could not be built for it. pip 21.0 will remove support for this functionality. A possible replacement is to fix the wheel build issue reported above. You can find discussion regarding this at https://github.com/pypa/pip/issues/8368.

but when updating to 21 the message just changes to:

  DEPRECATION: mindsdb was installed using the legacy 'setup.py install' method, because a wheel could not be built for it. A possible replacement is to fix the wheel build issue reported above. You can find discussion regarding this at https://github.com/pypa/pip/issues/8368.

and everything still works.

When is sdist going to be deprecated? Will it ever be?

I see no alternatives presented in here per say, is every library maintainer supposed to be build wheels for everything now? (Not an issue for me specifically, since I have access to resources to do so on dozens of different envs, but I assume for some maintainers this could prove difficult(?))

---

I’m fairly sure I’m misunderstanding something here, corrections would be appreciated.

Removing this from 21.0 because #9423 if this is the correct #8368 tab.

@sbidoul I have spited in two lines and this works without warning and using wheel. This is the output:

+ pip install 'graphviz==0.14.2' 'numpy==1.16.2'
Collecting graphviz==0.14.2
  Downloading graphviz-0.14.2-py2.py3-none-any.whl (18 kB)
Collecting numpy==1.16.2
  Downloading numpy-1.16.2.zip (5.1 MB)
Building wheels for collected packages: numpy
  Building wheel for numpy (setup.py): started
  Building wheel for numpy (setup.py): still running...
  Building wheel for numpy (setup.py): still running...
  Building wheel for numpy (setup.py): still running...
  Building wheel for numpy (setup.py): still running...
  Building wheel for numpy (setup.py): still running...
  Building wheel for numpy (setup.py): still running...
  Building wheel for numpy (setup.py): finished with status 'done'
  Created wheel for numpy: filename=numpy-1.16.2-cp38-cp38-linux_x86_64.whl size=10612313 sha256=136c953f8560dd57cdd651b2b9c8af3b3454d00098f8b26c3ba223626f32e82e
  Stored in directory: /tmp/pip-ephem-wheel-cache-79x7lajv/wheels/c7/1a/26/d24add95f4ea99e790c3a8b339fb5b46b66f9849142d1b93c5
Successfully built numpy
Installing collected packages: numpy, graphviz
Successfully installed graphviz-0.14.2 numpy-1.16.2
+ pip install 'scipy==1.2.1'
Collecting scipy==1.2.1
  Downloading scipy-1.2.1.tar.gz (23.1 MB)
Building wheels for collected packages: scipy
  Building wheel for scipy (setup.py): started
  Building wheel for scipy (setup.py): still running...
  Building wheel for scipy (setup.py): still running...
  Building wheel for scipy (setup.py): still running...
  Building wheel for scipy (setup.py): still running...
  Building wheel for scipy (setup.py): still running...
  Building wheel for scipy (setup.py): still running...
  Building wheel for scipy (setup.py): still running...
  Building wheel for scipy (setup.py): still running...
  Building wheel for scipy (setup.py): still running...
  Building wheel for scipy (setup.py): still running...
  Building wheel for scipy (setup.py): still running...
  Building wheel for scipy (setup.py): still running...
  Building wheel for scipy (setup.py): still running...
  Building wheel for scipy (setup.py): still running...
  Building wheel for scipy (setup.py): still running...
  Building wheel for scipy (setup.py): still running...
  Building wheel for scipy (setup.py): finished with status 'done'
  Created wheel for scipy: filename=scipy-1.2.1-cp38-cp38-linux_x86_64.whl size=55957602 sha256=84f7c97fc95b6893c1924f8d90851b69cc53de3ebb6d536d049b581cbb293532
  Stored in directory: /tmp/pip-ephem-wheel-cache-cr9csfcz/wheels/0a/51/83/29c07027bd3a7ad5cd422b3925ce6a1b8d8232f3beba759588
Successfully built scipy
Installing collected packages: scipy
Successfully installed scipy-1.2.1

@pfmoore - Forgot to answer your question, but basically: I get bug reports. I think it’s not easy to keep up with changes in build systems, but enough people do keep up and let me know of important ones through my issue tracker. Similarly, many build systems are compatible with setuptools to some extend, but only if the documented rules are followed. So, if my use doesn’t follow those to the letter, it will break one of the many other build systems out there, and someone will bug me.

As for learning about in-tree backends, I got that information right here. 😃