setuptools: [BUG] Setuptools 69.0.3 `setup.py develop` generates a .egg-link file with underscores in the name component

setuptools version

69.0.3

Python version

CPython 3.11.7

OS

Ubuntu 22.04.03 LTS

Additional environment information

No response

Description

Happy New Year! 🎉

I was debugging pip’s CI which has been red ever since the release of Setuptools 69.0.3. Most of them are easily fixed by removing the underscore -> dash normalization assumption, however, there is one place where leaving the underscores intact causes issues: legacy editable installs.

As documented in the Setuptools documentation, the distribution name part of egg filenames should be normalized by safe_name():

The “name” and “version” should be escaped using the to_filename() function provided by pkg_resources, after first processing them with safe_name() and safe_version() respectively. These latter two functions can also be used to later “unescape” these parts of the filename. (For a detailed description of these transformations, please see the “Parsing Utilities” section of the pkg_resources manual.)

Setuptools does not honour this.[^1] This is actually fine in most situations as far as I can tell since there are modern ways for pip to discover installed distributions that don’t rely on eggs, but setup.py develop does not generate this modern metadata. Thus, pip falls back to searching sys.path for a .egg-link file to determine whether a distribution is editably installed. Pip assumes the egg link name will be normalized by safe_name() so this logic returns a false negative despite version_pkg being editably installed in fact.

If I’m being honest, I have no idea whose problem this is, but this does mean for projects that do not implement PEP 518 and have underscores in their name will not be recognized as editably installed. If this is better transferred to the pip repository, please let me know!

[^1]: Well, if you read the documentation closely, apparently you’re supposed to pass the made safe name to pkg_resources.to_filename() which would turn the dash back into an underscore but setuptools does not seem to do this anyway /shrug

Expected behavior

See above.

How to Reproduce

1. Create an environment with pip<24, setuptools==69.0.3 and wheel.
2. Create a new directory for testing and create a setup.py representing a version_pkg:

# setup.py
import setuptools; setuptools.setup(name="version_pkg")

1. Run pip install -e . in the directory
2. Run pip freeze or pip list and observe that pip doesn’t realize it’s an editable install
3. Observe that the egg-link file has a underscore
4. Rerun this with Setuptools 69.0.2 and observe that pip can recognize that version_pkg is installed as an editable

Output

ichard26@asus-ubuntu:~/dev/oss/pip/temp/pip-test-package$ pip install setuptools==69.0.3 -q
ichard26@asus-ubuntu:~/dev/oss/pip/temp/pip-test-package$ python ../generate.py 
ichard26@asus-ubuntu:~/dev/oss/pip/temp/pip-test-package$ pip install -e .
Obtaining file:///home/ichard26/dev/oss/pip/temp/pip-test-package
  Preparing metadata (setup.py) ... done
Installing collected packages: version_pkg
  Running setup.py develop for version_pkg
Successfully installed version_pkg-0.1
ichard26@asus-ubuntu:~/dev/oss/pip/temp/pip-test-package$ pip freeze
version_pkg==0.1
ichard26@asus-ubuntu:~/dev/oss/pip/temp/pip-test-package$ pip list
Package     Version
----------- -------
pip         23.3.2
setuptools  69.0.3
version_pkg 0.1
wheel       0.42.0
ichard26@asus-ubuntu:~/dev/oss/pip/temp/pip-test-package$ ls -gha ~/dev/oss/pip/venv/lib/python3.11/site-packages/ | grep "version"
-rw-rw-r--  1 ichard26   50 Jan  1 16:56 version_pkg.egg-link