poetry: Path dependencies with setup.py cannot be installed if setup.py imports `numpy` (or more general, imports any site-packages)
- I am on the latest Poetry version.
- I have searched the issues of this repo and believe that this is not a duplicate.
- If an exception occurs when executing a command, I executed it again in debug mode (
-vvv
option).
- OS version and name: Windows 10
- Poetry version: 1.1.4
Issue
I have searched in here and found #2638 is similar but not the same problem as I am facing. I have also tried a possible solution of [build-system].requires
and consulted its documentation (as specified in #744) but no success. Here is a snapshot of the error message:
C:\Users\XXX\Desktop\test-module>poetry update -vvv
Updating dependencies
Resolving dependencies...
PackageInfoError
Unable to determine package info for path: C:\Users\XXX\Desktop\test-module\lib\my-dev-module
Fallback egg_info generation failed.
Command C:\Users\XXX\AppData\Local\Temp\tmpg5w9jw0j\.venv\Scripts\python.exe setup.py egg_info errored with the following return code 1, and output:
Traceback (most recent call last):
File "setup.py", line 11, in <module>
import numpy
ModuleNotFoundError: No module named 'numpy'
at ~\.poetry\lib\poetry\inspection\info.py:503 in _pep517_metadata
499│ venv.run("python", "setup.py", "egg_info")
500│ return cls.from_metadata(path)
501│ except EnvCommandError as fbe:
502│ raise PackageInfoError(
→ 503│ path, "Fallback egg_info generation failed.", fbe
504│ )
505│ finally:
506│ os.chdir(cwd.as_posix())
507│
I will reuse its structure and show how the problem is reproduced.
The project structure is something like
test-module/
+ pyproject.toml
+ testmodule/
+ __init__.py
+ lib/
+ my-dev-module/
+ setup.py
+ mydevmodule/
+ __init__.py
The toml file is
[tool.poetry]
name = "testmodule"
version = "0.1.0"
[tool.poetry.dependencies]
python = "^3.8"
numpy = "^1.19.1"
mydevmodule = {path = "./lib/my-dev-module", develop = true}
[build-system]
requires = ["numpy>=1.19.1,<1.20.0","poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"
And ./lib/my-dev-module/setup.py
is like
from distutils.core import setup
import numpy
if __name__ == "__main__":
setup(
name="mydevmodule",
version="0.2",
description="",
author="me",
)
The setup() in setup.py is a placeholder and the script does not get executed to setup(). It erred out at line 2 when importing numpy. In my actual application, module numpy is used to provide its C library for non-Python extension.
My question is: why does it fail at importing numpy? Shouldn’t specifying [build-system].requires
fix this? If not, I got that to fix this problem requires a [build-system].requires
and a pyproject.toml in mydevmodule
but that is not an ideal solution. Is there any other ways to specify the installation time requirements of a path-dependency module? Note that I have put numpy in both the dependencies and build-system requirement of the top-level project testmodule
.
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 1
- Comments: 16 (6 by maintainers)
There are currently no plans to drop the support for non-pyproject.toml projects. Furthermore PEP517 says “If the pyproject.toml file is absent, or the build-backend key is missing, the source tree is not using this specification, and tools should revert to the legacy behaviour of running setup.py (either directly, or by implicitly invoking the setuptools.build_meta:legacy backend).”
@hyliu1989 You can add the minimal
pyproject.toml
without changing anything else. No modification tosetup.py
. Developers of the project can still usepython setup.py develop
or whatever other setuptools commands they need. There should be no drawbacks when done correctly. But the big advantage of making sure that numpy is available when pip (or poetry or any other build frontend) need to build that project.@hyliu1989
Looks to me like you might need a
./lib/my-dev-module/pyproject.toml
file such as:Although it might get complicated for the editable installation…