pdm: slow pdm lock example

I have searched the issue tracker and believe that this is not a duplicate.

Make sure you run commands with -v flag before pasting the output.

Steps to reproduce

$ mkdir stackstac
$ cd stackstac
$ git clone git@github.com:gjoseph92/stackstac.git .
$ git rev-parse HEAD
e30ecf233ee42c1c98fcfbc5178ff105ba60760e
$ pdm --version
PDM, version 2.8.2
$ time pdm --lock

Actual behavior

I actually don’t know how long this takes - it has been about forty minutes so far. But it’s a long time.

Expected behavior

Faster locking.

Environment Information

# Paste the output of `pdm info && pdm info --env` below:
PDM version:
  2.8.2
Python Interpreter:
  /home/dch/stackstac/.venv/bin/python (3.10)
Project Root:
  /home/dch/stackstac
Local Packages:

{
  "implementation_name": "cpython",
  "implementation_version": "3.10.12",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "5.15.90.1-microsoft-standard-WSL2",
  "platform_system": "Linux",
  "platform_version": "#1 SMP Fri Jan 27 02:56:13 UTC 2023",
  "python_full_version": "3.10.12",
  "platform_python_implementation": "CPython",
  "python_version": "3.10",
  "sys_platform": "linux"
}

I actually found this one by looking for examples of poetry being slow, in the hope of finding something interesting that could be improved there. stackstac switched from poetry to pdm “because Poetry was taking multiple hours to lock dependencies”.

However today it seems to be pdm that is taking a considerable time to lock dependencies, so I thought you might appreciate the example.

About this issue

Original URL
State: open
Created 10 months ago
Comments: 26 (13 by maintainers)

Most upvoted comments

Well, the resolution failure can be also observed on pip using the following requirements.txt:

pandas
pystac    # removing this will get a working resolution
pystac-client
sat-stac

It seems the pystac requirement somehow crashes the tree and prevent it from backtracking further. Because in the whole process till it reaches to a failure, pandas 1.* were never tried.

So it is a certain issue with resolvelib, and the fix should also happen there.

frostming on Aug 21, 2023

But if the resolver fails to resolve some dependencies that becomes successful after making the range more specific, this would be a bug.

yes, that’s where we got to. The thread got a bit long but the course it has taken is

here is a set of requirements for which pdm is unable to make a decision in reasonable time
here is a subset of those requirements that pdm says is impossible to solve, although a solution is available
adding more specific pins to the second set does allow pdm to find that solution

I guess it’s plausible that if you fix the bug associated with the second set of requirements, then the failure-to-reach-any-decision of the original report may also go away.

dimbleby on Aug 20, 2023

Thanks for insisting 🙂 And thanks for the initial report.

If I remove pystac from the dependencies, it solves after a few minutes, ending with this:

pandas 1.3.5
(pystac 1.8.3)
pystac-client 0.6.1
python-dateutil 2.7.5
(sat-stac 0.4.1)

So same as Poetry. Not sure why adding pystac to the deps gives the above error message.

pawamoy on Aug 19, 2023

https://gist.github.com/dimbleby/f4599ffb2429400fc1c9963b632ccc82

dimbleby on Aug 19, 2023

A compatible solution does exist, but pdm is failing to find it in reasonable time. (I still don’t know how long it takes, the command is still running).

If you’re OK with “eventually” then fair enough. But I’d think that fast is better than slow.

dimbleby on Aug 19, 2023