astropy: Warnings about ignored packages
Description
With the switch from setup.cfg
to pyproject.toml
(#15247) the installation log now reports many warnings about ignored packages, e.g.:
check.warn(importable)
/tmp/simon/pip-build-env-990sh2z5/overlay/lib/python3.11/site-packages/setuptools/command/build_py.py:204: _Warning: Package 'astropy.convolution.src' is absent from the `packages` configuration.
!!
********************************************************************************
############################
# Package would be ignored #
############################
Python recognizes 'astropy.convolution.src' as an importable package[^1],
but it is absent from setuptools' `packages` configuration.
This leads to an ambiguous overall configuration. If you want to distribute this
package, please make sure that 'astropy.convolution.src' is explicitly added
to the `packages` configuration field.
Alternatively, you can also rely on setuptools' discovery methods
(for example by using `find_namespace_packages(...)`/`find_namespace:`
instead of `find_packages(...)`/`find:`).
You can read more about "package discovery" on setuptools documentation page:
- https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
If you don't want 'astropy.convolution.src' to be distributed and are
already explicitly excluding 'astropy.convolution.src' via
`find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
you can try to use `exclude_package_data`, or `include-package-data=False` in
combination with a more fine grained `package-data` configuration.
You can read more about "package data files" on setuptools documentation page:
- https://setuptools.pypa.io/en/latest/userguide/datafiles.html
[^1]: For Python, any directory (with suitable naming) can be imported,
even if it does not contain any `.py` files.
On the other hand, currently there is no concept of package data
directory, all directories are treated like packages.
********************************************************************************
and same for all data
directories and other directories that don’t have an __init__.py
file.
It’s related to https://setuptools.pypa.io/en/latest/userguide/package_discovery.html and the fact that setuptools now prefers the use of a src-layout (though I don’t know why switching from setup.cfg
to pyproject.toml
triggers this behavior).
I had this issue with other packages and solved that by using a src-layout but it seems astropy is not ready for that yet.
However we should find a solution and make sure that those directories are bundled in wheels/sdist.
I think the solution is to use namespace=True (i.e. find_namespace_packages
vs find_packages
) but this requires more investigation to be sure.
About this issue
- Original URL
- State: closed
- Created 9 months ago
- Reactions: 1
- Comments: 17 (17 by maintainers)
While I find that the ruff style changes are often for the better, I agree with @bsipocz that they also have caused a lot of extra work, and very much agree that without a really good cause we should not do further refactors that touch most of astropy.
Anyway, thanks @pllim for what looks like a solution to the actual problem!!!
It’s documented in https://setuptools.pypa.io/en/latest/userguide/datafiles.html#subdirectory-for-data-files so seems a valid use case to me.
Actually, it’s not even just scipy and numpy, I have to start digging very deeply to find any of the core scientific python projects that do src. Please don’t refactor yet again for the sake of some axillary tools, but rather get the focus back on fixing actual bugs and issues. The recent noise and added code conflicts are not at all helpful, especially since the whole styling refactor was done a while back with the promise that these types of conflicts are for the past. While in fact, it was never this bad before.
There are plenty of actual bugs and ideas for improvements to choose from, many are around for years and are still just as relevant as they have ever been. Let’s fix those instead.
It’s only when looking at the full compilation/installation log (from setuptools), which is not shown by default when using tox.
Neither numpy nor scipy use the
src
layout; not sure why we should.(Sorry to be a bit grumpy, but for my taste ther recent focus on “esthetics” has rather distracted from actually getting new or improved functionality – and the result seems, if anything, more things that break CI - I certainly don’t envy all what @pllim has to do to keep the show running.)
I’d rather have #15247 reverted than let it lead to yet more needless churn… (But hopefully this has a simple solution…)