pip: pip wheel . does not follow symlinks whereas python setup.py bdist_wheel does

I now think this issue or variants of it arise in any situation where

• setup.py / pyproject.toml is not at the root of the project
• the build depends on resources outside of the setup.py / pyproject.toml subtree

I found three variants:

1. build needs resources that are symlinks to files/directories outside of the subtree (the case which prompted this issue)
2. build needs the git repository (eg when using setuptools_scm), and .git/ is in a parent directory not copied to the temp dir by pip
3. build relies on the subdirectory name (somewhat exotic maybe, yet I have such as case where I want to create a custom build backend and part of the metadata depends on the subdirectory name)

Since pip copies the subtree to a temporary location before building, when running pip install . or pip wheel . resources that reside out of the subtree are not available for building.

I think this happens both for pep517 and legacy builds.

Note that when running pip install git+http://g.c/repo/project#egg=project&subdirectory=subdir it works fine because the whole project is available (I think no additional copy takes place in that case).

So while it could be possible to fix this particular issue (case 1.) with some strategy to preserve symlinks when copying to the temporary directory, this would not solve 2. nor 3.

I’m not sure if pep517 build isolation mandates working on a temporary copy of the source tree.

Maybe a possible approach could be to have a way for build backends to declare that they are safe for in-place builds (in an isolated build environment, but in the same source tree). Build backend would then have a way to promise to front ends they will not modify the source tree. And pip could use that information to decide to build in place without copying to a temporary directory.

Do you think it’s a track worth investigating further?

@RonnyPfannschmidt I’m not sure that’s true, since we’re going to be executing a setup.py anyways after we copy it, I doubt there’s anything you can do with symlinks you couldn’t just do in the Python file itself.