pip-tools: pip==23.3 breaking "extras" behavior

Using pip==23.3.x can result in invalid requirements.txt for projects that declare extras. A change in pip==23.3.0 introduced the “canonicalization” of extras names:

• https://github.com/pypa/pip/pull/12002/files

The canonicalization leads to this outcome:

• requirements.in: sqlalchemy[postgresql_psycopg2binary]
• requirements.txt: sqlalchemy[postgresql-psycopg2binary]

For many cases, this probably isn’t an issue. psycopg2-binary still makes it into requirements.txt as an individual entry.

For more “niche” cases, the extras still carry some significance. eg. When using rules_python under the Bazel build system, the extras must properly resolve to build out a dependency graph.

Environment Versions

1. OS Type: Linux
2. Python version: 3.10.9
3. pip version: 23.3.0
4. pip-tools version: 7.3.0

Steps to replicate

1. Create a simple requirements.in file.

sqlalchemy[postgresql_psycopg2binary]==1.4.47

1. Create a virtual environment, install pip-tools and the previous major pip version, run pip-compile.

python -m venv .venv
. ./.venv/bin/activate
pip install pip-tools==7.3.0 pip==22.3.1
pip-compile --no-strip-extras

1. Observe that the resulting requirement.txt file lists sqlalchemy[postgresql_psycopg2binary] (with an underscore).

greenlet==3.0.0
    # via sqlalchemy
psycopg2-binary==2.9.9
    # via sqlalchemy
sqlalchemy[postgresql_psycopg2binary]==1.4.47
    # via -r requirements.in

1. On the same virtual environment, upgrade to pip>=23.3, run pip-compile.

pip install pip==23.3.0
pip-compile --no-strip-extras

1. Observe that the resulting requirement.txt file lists sqlalchemy[postgresql-psycopg2binary] (with a hyphen).

  WARNING: sqlalchemy 1.4.47 does not provide the extra 'postgresql-psycopg2binary'

greenlet==3.0.0
    # via sqlalchemy
psycopg2-binary==2.9.9
    # via sqlalchemy
sqlalchemy[postgresql-psycopg2binary]==1.4.47
    # via
    #   -r requirements.in
    #   sqlalchemy

Expected result

The valid input extras should still be valid in the output.

Actual result

The output extras will not resolve.