setuptools: find_packages() doesn't find PEP 420 packages
Originally reported by: gwideman (Bitbucket: gwideman, GitHub: gwideman)
#!python
setup(...
packages=find_packages(..)
)
on the developer machine will fail to find packages that lack a __init__.py file, as is allowed in Python 3.3. However, such packages listed explicitly: packages=[‘mypkg’] do appear to get included and later installed.
Note: When testing this, before each test be sure to delete all generated metadata, including that which setup may previously have placed in the original source directory, as it seems that setup may use metadata created on a previous run in order to include files.
This is part of a general problem reported in issue #83, but I’ve logged it separately as it’s specifically about setuptools.
About this issue
- Original URL
- State: closed
- Created 11 years ago
- Reactions: 12
- Comments: 95 (30 by maintainers)
Commits related to this issue
- Remove package module for namespace package 'vr' and specify it explicitly because find_packages will not find PEP 420 namespace packages per pypa/setuptools#97. — committed to yougov/vr.cli by jaraco 8 years ago
- Add test capturing new expectation. Ref #97. — committed to pypa/setuptools by jaraco 8 years ago
- Add comment explaining that the namespace package module may be omitted, but that the namespace package must still be declared. Ref #97. — committed to pypa/setuptools by jaraco 8 years ago
- Ensure that namespace packages are implicitly declared as packages. Ref #97. — committed to pypa/setuptools by jaraco 8 years ago
- Revert "Remove package module for namespace package 'vr' and specify it explicitly because find_packages will not find PEP 420 namespace packages per pypa/setuptools#97." This reverts commit 6f33bda9... — committed to yougov/vr.cli by jaraco 8 years ago
- Fixed workaround for https://github.com/pypa/setuptools/issues/97 to search recursively — committed to IwanVosloo/reahl by IwanVosloo 7 years ago
- Add namespace_packages parameter to find_packages The user must pass any implicit (PEP 420) namespace packages in this parameter. Packages in this list will be recognized as such, despite not having ... — committed to estan/setuptools by deleted user 7 years ago
- For issue #97 find_packages finds namespace packages. — committed to illume/setuptools by illume 7 years ago
- Don't use `find_packages` `find_packages` has problems with packages without an `__init__.py` file (this is the short version, see <https://github.com/pypa/setuptools/issues/97> for more information)... — committed to kfl/Staffeli by kfl 7 years ago
- Reflect find_packages behavior in doc find_packages doesnt find PEP420 packages but the documentation suggests that it does. see: pypa/setuptools#97 — committed to dylanjw/setuptools by dylanjw 6 years ago
- fix #97 make find_packages() find also PEP420 namespace packages when supported — committed to coldrye-collaboration/setuptools by silkentrance 6 years ago
- fix #97 introduce find_packages_ns() for also finding PEP420 namespace packages — committed to coldrye-collaboration/setuptools by silkentrance 6 years ago
- fix #97 introduce find_packages_ns() for also finding PEP420 namespace packages — committed to coldrye-collaboration/setuptools by silkentrance 6 years ago
- fix #97 introduce find_packages_ns() for also finding PEP420 namespace packages — committed to coldrye-collaboration/setuptools by silkentrance 6 years ago
- fix #97 introduce find_packages_ns() for also finding PEP420 namespace packages — committed to coldrye-collaboration/setuptools by silkentrance 6 years ago
- fix #97 introduce find_packages_ns() for also finding PEP420 namespace packages — committed to coldrye-collaboration/setuptools by silkentrance 6 years ago
- fix #97 introduce find_packages_ns() for also finding PEP420 namespace packages — committed to coldrye-collaboration/setuptools by silkentrance 6 years ago
- fix #97 introduce find_packages_ns() for also finding PEP420 namespace packages — committed to coldrye-collaboration/setuptools by silkentrance 6 years ago
- fix #97 introduce find_packages_ns() for also finding PEP420 namespace packages — committed to coldrye-collaboration/setuptools by silkentrance 6 years ago
- fix #97 introduce find_packages_ns() for also finding PEP420 namespace packages — committed to coldrye-collaboration/setuptools by silkentrance 6 years ago
Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):
I believe this issue is more than a minor one and should be addressed sooner than later.
@pganssle How do I use
find_packages_nsfromsetup.cfg? It’s not obvious from your commit.My recommendation for now is that package maintainers continue to use
__init__.pyfiles in their namespace packages to mark the folder as a package for find_packages. Either that or enumerate the packages explicitly and not usefind_packages. I haven’t made any advancement on the branch, but I do agree, what you propose would be preferable.I’ve flagged this issue as help-wanted because I simply don’t have time to address all of these issues on my own. Please feel free to grab the work I’ve done and advance it, or start again from the current master.
Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):
I’d like to say that Setuptools should support PEP 420 packages. The harder question is how should
find_packagesdiscover such packages? PEP 420 allows importing of any importable filename ending in .py in any directory insys.path. However,sys.pathisn’t relevant when building distributions and discovering packages.I imagine setuptools could include all suitable directories, possibly omitting explicitly excluded paths. This change would be backwards-incompatible, but could provide packagers with options to produce compatible dists.
I considered using Python’s import machinery to find packages, but I don’t believe the import machinery provides any discovery tools.
The other thing to consider - if setuptools supports this mode, it will not work on other versions of Python, so will require Python 3.3+.
This gives me an idea - perhaps setuptools can use the package metadata to determine when PEP 420 package discovery is enabled. It could inspect the Trove classifiers and if Python versions are specified and only include versions >= 3.3, the feature would be enabled.
That technique would be somewhat implicit, but properly-documented, it could provide an elegant way to allow a packager to both include PEP 420 packages and declare the Python requirement at the same time.