poetry: Poetry 1.8.0 unable to lock files (worked in 1.7.1)
Description
Running poetry lock with latest poetry 1.8.0 seems broken with:
Sonatype Nexus Repository OSS 3.52.0-01
Updating dependencies
Resolving dependencies... (12.6s)
404 Client Error: Not Found for url: https://myuser:mypass@nexus.company.net/repository/pypi-blessed/packages/django-stubs-ext/4.2.7/django_stubs_ext-4.2.7-py3-none-any.whl#sha256=a8ba463124b3de17fe11c4a9cf2080511ec9d5da8bf890b20f5e5a614dfa266c
[[tool.poetry.source]]
name = "nexus"
url = "https://myuser:mypass@nexus.company.net/repository/pypi-blessed/simple"
priority = "primary"
(.venv) ➜ curl -v https://myuser:mypass@nexus.company.net/repository/pypi-blessed/packages/django-stubs-ext/4.2.7/django_stubs_ext-4.2.7-py3-none-any.whl\#sha256\=a8ba463124b3de17fe11c4a9cf2080511ec9d5da8bf890b20f5e5a614dfa266c
* using HTTP/1.1
* Server auth using Basic with user 'myuser'
> GET /repository/pypi-blessed/packages/django-stubs-ext/4.2.7/django_stubs_ext-4.2.7-py3-none-any.whl HTTP/1.1
> Host: nexus.mycompany.net
> Authorization: Basic ....
> User-Agent: curl/8.4.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx/1.24.0
< Date: Mon, 26 Feb 2024 08:56:04 GMT
< Content-Type: application/zip
< Content-Length: 8864
< Last-Modified: Wed, 06 Dec 2023 06:59:37 GMT
< ETag: "{SHA1{9192aaf7a657752fa183da289f64e9a1c39d6e81}}"
<
Warning: Binary output can mess up your terminal. Use "--output -" to tell
Workarounds
no known workarounds except downgrade.
tried: poetry config http-basic.nexus <username> <password>
Poetry Installation Method
pipx
Operating System
MacOSX
Poetry Version
Poetry (version 1.8.0)
Poetry Configuration
cache-dir = "/Users/myuser/Library/Caches/pypoetry"
experimental.system-git-client = false
installer.max-workers = null
installer.modern-installation = true
installer.no-binary = null
installer.parallel = true
keyring.enabled = true
repositories.nexus.url = "https://myuser:mypass@nexus.mycompany/repository/pypi-blessed/simple"
solver.lazy-wheel = true
virtualenvs.create = true
virtualenvs.in-project = true
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.no-setuptools = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}/virtualenvs" # /Users/myuser/Library/Caches/pypoetry/virtualenvs
virtualenvs.prefer-active-python = false
virtualenvs.prompt = "{project_name}-py{python_version}"
warnings.export = false
Python Sysconfig
No response
Example pyproject.toml
[tool.poetry]
name = "bugreport"
version = "0.0.0"
description = "Bla"
authors = ["bla"]
[tool.poetry.dependencies]
python = ">=3.11, <4"
django-stubs-ext = "==4.2.7"
[[tool.poetry.source]]
name = "nexus"
url = "https://myuser:mypass@nexus.mycompany.net/repository/pypi-blessed/simple"
priority = "primary"
Poetry Runtime Logs
(.venv) ➜ poetrylock poetry -vvv lock
Adding repository nexus (https://myuser:mypass@nexus.mycompany.net/repository/pypi-blessed/simple) and setting it as primary
Deactivating the PyPI repository
Using virtualenv: /Users/myuser/projects/myproject/.venv
Updating dependencies
Resolving dependencies...
1: fact: bugreport is 0.0.0
1: derived: bugreport
1: fact: bugreport depends on django-stubs-ext (==4.2.7)
1: selecting bugreport (0.0.0)
1: derived: django-stubs-ext (==4.2.7)
Creating new session for myuser:mypass@nexus.mycompany.net
[urllib3:urllib3.connectionpool] Starting new HTTPS connection (1): nexus.mycompany.net:443
[urllib3:urllib3.connectionpool] https://nexus.mycompany.net:443 "GET /repository/pypi-blessed/simple/django-stubs-ext/ HTTP/1.1" 200 2736
Source (nexus): 1 packages found for django-stubs-ext 4.2.7
begin fetching content length
initial bytes request: bytes=-10000
[urllib3:urllib3.connectionpool] https://nexus.mycompany.net:443 "GET /repository/pypi-blessed/packages/django-stubs-ext/4.2.7/django_stubs_ext-4.2.7-py3-none-any.whl HTTP/1.1" 404 1824
1: Version solving took 0.172 seconds.
1: Tried 1 solutions.
ValueError
Package('django-stubs-ext', '4.2.7') is not in list
at ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/repositories/legacy_repository.py:66 in package
62│ Note that this will be cached so the subsequent operations
63│ should be much faster.
64│ """
65│ try:
→ 66│ index = self._packages.index(Package(name, version))
67│
68│ return self._packages[index]
69│ except ValueError:
70│ package = super().package(name, version, extras)
The following error occurred when trying to handle this error:
Stack trace:
34 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/cleo/application.py:327 in run
325│
326│ try:
→ 327│ exit_code = self._run(io)
328│ except BrokenPipeError:
329│ # If we are piped to another process, it may close early and send a
33 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/console/application.py:190 in _run
188│ self._load_plugins(io)
189│
→ 190│ exit_code: int = super()._run(io)
191│ return exit_code
192│
32 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/cleo/application.py:431 in _run
429│ io.input.interactive(interactive)
430│
→ 431│ exit_code = self._run_command(command, io)
432│ self._running_command = None
433│
31 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/cleo/application.py:473 in _run_command
471│
472│ if error is not None:
→ 473│ raise error
474│
475│ return terminate_event.exit_code
30 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/cleo/application.py:457 in _run_command
455│
456│ if command_event.command_should_run():
→ 457│ exit_code = command.run(io)
458│ else:
459│ exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED
29 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/cleo/commands/base_command.py:117 in run
115│ io.input.validate()
116│
→ 117│ return self.execute(io) or 0
118│
119│ def merge_application_definition(self, merge_args: bool = True) -> None:
28 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/cleo/commands/command.py:61 in execute
59│
60│ try:
→ 61│ return self.handle()
62│ except KeyboardInterrupt:
63│ return 1
27 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/console/commands/lock.py:55 in handle
53│ self.installer.lock(update=not self.option("no-update"))
54│
→ 55│ return self.installer.run()
56│
26 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/installation/installer.py:104 in run
102│ self.verbose(True)
103│
→ 104│ return self._do_install()
105│
106│ def dry_run(self, dry_run: bool = True) -> Installer:
25 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/installation/installer.py:241 in _do_install
239│ source_root=self._env.path.joinpath("src")
240│ ):
→ 241│ ops = solver.solve(use_latest=self._whitelist).calculate_operations()
242│ else:
243│ self._io.write_line("Installing dependencies from lock file")
24 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/puzzle/solver.py:71 in solve
69│ with self._progress(), self._provider.use_latest_for(use_latest or []):
70│ start = time.time()
→ 71│ packages, depths = self._solve()
72│ end = time.time()
73│
23 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/puzzle/solver.py:154 in _solve
152│
153│ try:
→ 154│ result = resolve_version(self._package, self._provider)
155│
156│ packages = result.packages
22 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/mixology/__init__.py:18 in resolve_version
16│ solver = VersionSolver(root, provider)
17│
→ 18│ return solver.solve()
19│
21 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/mixology/version_solver.py:175 in solve
173│ while next is not None:
174│ self._propagate(next)
→ 175│ next = self._choose_package_version()
176│
177│ return self._result()
20 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/mixology/version_solver.py:514 in _choose_package_version
512│ package = locked
513│
→ 514│ package = self._provider.complete_package(package)
515│
516│ conflict = False
19 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/puzzle/provider.py:489 in complete_package
487│ dependency_package = DependencyPackage(
488│ dependency,
→ 489│ self._pool.package(
490│ package.pretty_name,
491│ package.version,
18 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/repositories/repository_pool.py:204 in package
202│ for repo in self.repositories:
203│ try:
→ 204│ return repo.package(name, version, extras=extras)
205│ except PackageNotFound:
206│ continue
17 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/repositories/legacy_repository.py:70 in package
68│ return self._packages[index]
69│ except ValueError:
→ 70│ package = super().package(name, version, extras)
71│ package._source_type = "legacy"
72│ package._source_url = self._url
16 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/repositories/cached_repository.py:75 in package
73│ extras: list[str] | None = None,
74│ ) -> Package:
→ 75│ return self.get_release_info(canonicalize_name(name), version).to_package(
76│ name=name, extras=extras
77│ )
15 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/repositories/cached_repository.py:52 in get_release_info
50│ return PackageInfo.load(self._get_release_info(name, version))
51│
→ 52│ cached = self._release_cache.remember(
53│ f"{name}:{version}", lambda: self._get_release_info(name, version)
54│ )
14 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/cache.py:147 in remember
145│ value = self.get(key)
146│ if value is None:
→ 147│ value = callback() if callable(callback) else callback
148│ self.put(key, value, minutes)
149│ return value
13 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/repositories/cached_repository.py:53 in <lambda>
51│
52│ cached = self._release_cache.remember(
→ 53│ f"{name}:{version}", lambda: self._get_release_info(name, version)
54│ )
55│
12 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/repositories/legacy_repository.py:123 in _get_release_info
121│ yanked = page.yanked(name, version)
122│
→ 123│ return self._links_to_data(
124│ links,
125│ PackageInfo(
11 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/repositories/http_repository.py:360 in _links_to_data
358│
359│ # drop yanked files unless the entire release is yanked
→ 360│ info = self._get_info_from_links(links, ignore_yanked=not data.yanked)
361│
362│ data.summary = info.summary
10 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/repositories/http_repository.py:310 in _get_info_from_links
308│ return self._get_info_from_metadata(
309│ universal_python3_wheel
→ 310│ ) or self._get_info_from_wheel(universal_python3_wheel)
311│
312│ if universal_python2_wheel:
9 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/repositories/http_repository.py:123 in _get_info_from_wheel
121│ try:
122│ package_info = PackageInfo.from_metadata(
→ 123│ metadata_from_wheel_url(link.filename, link.url, self.session)
124│ )
125│ except HTTPRangeRequestUnsupported:
8 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/inspection/lazy_wheel.py:77 in metadata_from_wheel_url
75│ # After context manager exit, wheel.name will point to a deleted file path.
76│ # Add `delete_backing_file=False` to disable this for debugging.
→ 77│ with LazyWheelOverHTTP(url, session) as lazy_file:
78│ metadata_bytes = lazy_file.read_metadata(name)
79│
7 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/inspection/lazy_wheel.py:289 in __enter__
287│ def __enter__(self: U) -> U:
288│ super().__enter__()
→ 289│ self._setup_content()
290│ return self
291│
6 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/inspection/lazy_wheel.py:357 in _setup_content
355│ if self._length is None:
356│ logger.debug("begin fetching content length")
→ 357│ self._length = self._fetch_content_length()
358│ logger.debug("done fetching content length (is: %d)", self._length)
359│ # Enable us to seek and write anywhere in the backing file up to this
5 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/inspection/lazy_wheel.py:508 in _fetch_content_length
506│ """
507│ initial_chunk_size = self._initial_chunk_length()
→ 508│ ret_length, tail = self._extract_content_length(initial_chunk_size)
509│
510│ # Need to explicitly truncate here in order to perform the write and seek
4 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/inspection/lazy_wheel.py:609 in _extract_content_length
607│ try:
608│ # Initial range request for just the end of the file.
→ 609│ file_length, tail = self._try_initial_chunk_request(initial_chunk_size)
610│ except HTTPError as e:
611│ resp = e.response
3 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/inspection/lazy_wheel.py:572 in _try_initial_chunk_request
570│
571│ self._request_count += 1
→ 572│ tail = self._session.get(self._url, headers=headers, stream=True)
573│ tail.raise_for_status()
574│
2 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/authenticator.py:267 in get
265│
266│ def get(self, url: str, **kwargs: Any) -> requests.Response:
→ 267│ return self.request("get", url, **kwargs)
268│
269│ def head(self, url: str, **kwargs: Any) -> requests.Response:
1 ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/authenticator.py:245 in request
243│ if resp.status_code not in STATUS_FORCELIST or is_last_attempt:
244│ if raise_for_status:
→ 245│ resp.raise_for_status()
246│ return resp
247│
HTTPError
404 Client Error: Not Found for url: https://myuser:mypass@nexus.mycompany.net/repository/pypi-blessed/packages/django-stubs-ext/4.2.7/django_stubs_ext-4.2.7-py3-none-any.whl#sha256=a8ba463124b3de17fe11c4a9cf2080511ec9d5da8bf890b20f5e5a614dfa266c
at ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/requests/models.py:1021 in raise_for_status
1017│ f"{self.status_code} Server Error: {reason} for url: {self.url}"
1018│ )
1019│
1020│ if http_error_msg:
→ 1021│ raise HTTPError(http_error_msg, response=self)
1022│
1023│ def close(self):
1024│ """Releases the connection back to the pool. Once this method has been
1025│ called the underlying ``raw`` object must not be accessed again.
(.venv) ➜ poetrylock
About this issue
- Original URL
- State: closed
- Created 4 months ago
- Reactions: 2
- Comments: 21 (12 by maintainers)
@matejsp Thanks a lot. 🙏 So my reproducer with curl was just not good enough. 🤦
Then, it’s clear that Nexus supports range requests but not negative offsets and instead of returning
not implemented
ormethod not allowed
it returnsnot found
. I’m confident now that #9030 will fix this.There is something wrong with Range: bytes=-10000 … that nexus does not accept and this changed in latest poetry
Curl with -I works. -I is just a HEAD response in CURL and it returns 200 (since it is not sending range bytes anyway).
However:
Same thing with
curl -I -H "Range: bytes=-10000" <link-to-file>
@david-nano makes sense. I suspect we will make a patch release for this as it might be an issue for a lot of folks.