pip: pip freeze with a hash

pip freeze --hash will be very useful.

Are you saying you think the entirety of pip freeze is out of scope for pip now?

Because if not, this seems like a very logical thing for pip. Not all of us use any current higher level tool, and it’s pip itself that introduced the possibility of having hashes in requirements files.

Without this feature it’s pretty unfeasible to generate those.

Saying “patches welcome” seems very reasonable, but closing not so much.

On Mon, Aug 6, 2018, 17:26 d❤vid notifications@github.com wrote:

Closed #4732 https://github.com/pypa/pip/issues/4732.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pypa/pip/issues/4732#event-1773030507, or mute the thread https://github.com/notifications/unsubscribe-auth/AAUIXkYXs7_gKTjNsPIAurvfoyr6k-Gkks5uOFIagaJpZM4Pduuk .

@andrewchambers perhaps instead of the slight barbs consider sending a PR?

This will generate a requirements.txt with hashes

pip-compile requirements.txt --generate-hashes

Note that this will directly modify existing requirements.txt file.

You can install pip-compile with pip install pip-tools

What about generating the lock file at install-time, like npm, yarn, pipenv, poetry, Cargo, and Conan do? (sorry if I missed any)

1. pip install -r requirements.txt --lock requirements.txt.lock
2. Commit requirements.txt.lock
3. Afterwards, invoke pip install -r requirements.txt.lock

On updates to requirements.txt, do the same steps.

This directly supports with the stated use case:

I am a Python developer with an existing requirements.txt file. I want to add hashes to the file, so that future installations are more secure.

but it avoids a lot of the extra work that is being described in #8519.

The fact this doesn’t exist is just terrible. I just wrote a workaround:

https://github.com/andrewchambers/mummipy

enjoy.

@pradyunsg @deveshks I am thinking about implementing this PR. I think that hashes should not be obtained from remote, and so must be computed at install time (maybe fallback from cache?) How about adding a new file to the wheel metadata that, similar to RECORD, that will contain the pre-installed hash of the package? Never did that but I saw that it is not a lot of diff and a simpler solution. This will allow pip freeze to quickly list hashes of installed packages, however there is no support of listing hashes of packages that were installed with older versions of pip. Another problem is that this behavior is different from the behavior of different tools that compute hashes for packages, which they usually get from a remote PYPI, which seems like an unsafe option overall.

I will start to work on this in the following days, let me know what you think about my idea 😄

I’ve labelled this issue as an “deferred till PR”.

This label is essentially for indicating that further discussion related to this issue should be deferred until someone comes around to make a PR. This does not mean that the said PR would be accepted - that decision has been deferred until the PR is made.

Is there any roadmap or concrete discussion about implementing the proposed -p / --pipfile option that may replace the -r option in the long run? I’m having a hard time to find this.

dang, this issue is almost five years old and still open/ambiguous.

@Julian note that it was the OP who closed the issue, not the pip developers. The option for someone to create a PR for this remains available to anyone interested in the feature.

You can get the hash from the cached wheel in ~/.cache/pip/wheels/

What you describe is exactly what pip is currently doing. The problem in this thread is the other way around: people are looking for a way to generate hashes from installed data, and the pip developers are trying to explain we don’t know how this can be done.

The approach suggested by @chrahunt in https://github.com/pypa/pip/issues/4732#issuecomment-657898593 is also valuable in a lot of situations. It has complexities to think through too, for instance when the install command is used to update an existing environment, and when pip decides it does not need to reinstall some already installed dependencies. In such cases we’d still need a way to obtain information about the hashes of installed distributions.

@NoahGorny WRT hashes:

• get the hash from the remote
• generate the hash locally
• verify the hashes match

Don’t trust remote, but don’t necessarily trust local either. Verify they match to give the user assurance that the right thing is installed. If they don’t match, generate an error; provide a way to force the install if they don’t match, but by default uninstall/rollback if they don’t match (or depending where/when you’re generating the hashes…don’t install to start with, which would be even better).

@NoahGorny cool you want to work on this.

You’ll also want to pay attention to the wheel cache. When installing something that has been built (e.g. an sdist) and cached as a wheel, we probably want the hash of the original sdist or direct url target, and not the hash of the wheel that we have in cache.

would still be good to have this directly via pip freeze instead of having to use other tooling; and pipenv and Pipfile comes with their own set of headaches.

I’ve updated the ticket description with the proposed solution (as I understand it). Note that Pipfile-based dependencies are usable today if you use pipenv.

It looks like pipenv is getting the hashes directly from the warehouse api

https://github.com/pypa/pipenv/blob/master/pipenv/utils.py#L468-L508

Is there at least some way to easily script this? E.g., can I loop over a pip freeze and somehow programmatically find the file I need to pass to pip hash?