poetry: JSONDecodeError on Python 3.10

BTW disabling poetry’s experimental new installer may be a workaround for now:

$ poetry config experimental.new-installer false

Reproducible with 3.10 on Linux & Windows as well.

Here’s how to fix it:

$ pip install -U cachecontrol

poetry cache clear --all pypi
poetry cache clear --all private

did the trick. For me which python version it run on seemed irrelevant.

So, the issue is resolved with the new version of cachecontrol==0.12.9 (it will be installed automatically).

Besides, when trying to reproduce this bug in a fresh container I see “SolverProblemError”, thus, after this update some of the issues with that error could be resolved as well.

How to reproduce:

docker run --rm -ti python:3.9 bash
pip install poetry CacheControl==0.12.8
poetry init -n
poetry config experimental.new-installer false
poetry add asyncpg


wait 900 secs


poetry update  <- fails
poetry update  <- succees

Reproduced with Poetry 1.1.10 installed on Python 3.9.5 (Ubuntu 21.04 system Python) using pipx.

poetry install started failing immediately when I switched project from Python 3.9.7 to 3.10.0rc2, which is due for final release tomorrow.

Works:

pyenv local 3.9.7
poetry env use $(pyenv which python)
poetry install

Fails:

pyenv local 3.10.0rc2
poetry env use $(pyenv which python)
poetry install

Poetry eats most of the stack trace. I see the following repeated once for each dependency:

  • Installing urllib3 (1.26.7): Failed

  JSONDecodeError

  Expecting value: line 1 column 1 (char 0)

  at /usr/lib/python3.9/json/decoder.py:355 in raw_decode
      351│         """
      352│         try:
      353│             obj, end = self.scan_once(s, idx)
      354│         except StopIteration as err:
    → 355│             raise JSONDecodeError("Expecting value", s, err.value) from None
      356│         return obj, end
      357│ 

Removing the Poetry cache with rm -r ~/.cache/pypoetry/cache/ or m -r ~/.cache/pypoetry/ does not improve the situation.

Disabling the experimental installer with poetry config experimental.new-installer false does resolve the issue, so that at least serves as an indication of what part of Poetry is failing.

This issue also occurs on Python3.6 with:

• Poetry version: 1.1.11
• Poetry-core version: 1.0.7
• OS version and name: Fedora 34

The two workarounds mentioned in this ticket work:

1. Clearing the poetry cache
2. Disabling the experimental installer using poetry config experimental.new-installer false

Hi all,

Wanted to update you that given error should gone with latest poetry==1.1.11 release.

With given release you don’t need to disable experimental new installer to install your dependencies with poetry at 3.10.0rc2 or 3.10.0.

I am glad I found this issue. I wouldn’t have known how to even debug this, since Poetry eats the whole traceback I only shows this JSONDecodeError.

Is there a way to print the traceback?

fresh install with install-poetry.py, deleting poetry.lock, and latest version of cachecontrol did not fix the issue for me, FYI

Hello everyone,

as others already mention this issue is solved by a new version of cachecontrol.

If you install poetry by the install-poetry.py scrip, the new version is installed automatically.

If you use the deprecated get-poetry.py script, it is not possible to update the cachecontrol version. get-poetry.py vendores the dependencies based on poetry’s lock file. get-poetry.py installs poetry 1.1 which still supports python2. The current version of cachecontrol doesn’t support python2 anymore, thus we cannot update the locked version in the 1.1 branch.

TL;DR;: Install poetry with the install-poetry.py to resolve this issue.

fin swimmer

Try installing CacheControl==0.12.6 or wait for https://github.com/ionrock/cachecontrol/pull/264

What happens:

1. After running poetry install at the first time every pypi.org response is cached. Besides, pypi.org sets cache control, its says max age for cached response is 900 secs
2. Wait some time (>900 secs)
3. Run poetry update. At this point cachecontrol trying to use the data from the cache, but its stale. So it asks pypi.org for new response, dumps it and returns to the requests library, pretending that its genuine. But when it dumps, it consumes buffer and dont rewind. So, when we do response.json(), requests lib is reading the buffer, but sinces its already consumed, what it gets is ‘’. Thats leading to JsonDecodeError.

What happens when you disable experimentail installer - the same thing as running poetry update second time. Cache was updated at step 3, so, installation process continues to the next stale item.

I can confirm that the issue is resolved by @hoefling’s suggestion.

BTW disabling poetry’s experimental new installer may be a workaround for now:

$ poetry config experimental.new-installer false

Anyone got this on 3.9.7? I get

 • Installing pycparser (2.20): Failed

  JSONDecodeError

  Expecting value: line 1 column 1 (char 0)

  at /usr/local/Cellar/python@3.9/3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/decoder.py:355 in raw_decode

Installed poetry using pip on Windows on 3.9.6, cannot get poetry install to work 😦 Tried all suggesitons in this post, with no luck

I use rm -rf ~/.cache/pypoetry/cache/.

The error is resolved with experimental.new-installer false every time.
With experimental.new-installer true, sometimes the error JSONDecodeError is raised and it works in next attempt without changing experimental.new-installer value. I guess it matches with ddc67cd’s comment.

I tested with:

$ ./working_venv/bin/poetry config --list | grep new-installer
experimental.new-installer = true

#! /bin/sh

VENV_DIR="venv"
PYTHON='python3.9'

rm -rf "${VENV_DIR}"
${PYTHON} -m venv "${VENV_DIR}"

"./${VENV_DIR}/bin/pip" install --upgrade pip setuptools poetry

"./${VENV_DIR}/bin/poetry" i

pyproject.toml file:

[tool.poetry]
name = "demo"
version = "0.1.0"
description = "demo"
authors = ["<demo@domain.tld>"]

[tool.poetry.dependencies]
Markdown = "^3.*"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

It can fail with other dependencies than Markdown.

There was a poetry.toml file during the tests but I don’t think it’s relevant:

[virtualenvs]
create = false

Came here to say it started happening out of nowhere on poetry 1.1.11 with python 3.8.12 on github actions ubuntu 20.04 when I added beautifulsoup4 as a dev dependency. No idea what makes that package special, I cleared the cache in the meantime.

If anyone else gets here: my issue was not solved by changing config, nor by a complete reinstall of poetry.

Solution: Simply delete and recreate my currently activated virtual environment.

Explanation: as @jcowles suggests above, my currently activated virtual environment python was semi-borked. It was, unbeknownst to me, dishing out a nice error message (but still actually functioning fine somehow):

% python
Error processing line 1 of /Users/james/miniconda/envs/mamod/lib/python3.7/site-packages/distutils-precedence.pth:

  Traceback (most recent call last):
    File "/Users/james/miniconda/envs/mamod/lib/python3.7/site.py", line 168, in addpackage
      exec(line)
    File "<string>", line 1, in <module>
  ModuleNotFoundError: No module named '_distutils_hack'

Remainder of file ignored
Python 3.7.13 (default, Mar 28 2022, 07:24:34)
[Clang 12.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

This looks to be related to https://github.com/pypa/setuptools/issues/2957. Presumably I had broken my venv via various installations.

So my advice if you get a JSONDecodeError with poetry is to simply execute python, and see if you get a weird error and, if so, just trash and recreate your venv.

@clintonroy - we could raise a feature request for poetry to handle the JSON parsing more gracefully? I believe this is done within this file: https://github.com/python-poetry/poetry/blob/ae03bdcdecbb00f2691dbd0f78c08fc010e87779/src/poetry/utils/env.py

The problem for me is that something was spewing “Cannot find the file specified” when launching python. Poetry initializes by running python and printing “sys.path” and then attempts to decode this using the json parser.

The JSON parser fails (loads) because the json string starts with “Cannot find the file specified”, and of course that’s not valid json.

Still a total mystery what is spewing that string though

I run into this issue on Python 3.8.8 as well.

Can someone provide steps to reproduce this, trying to addd bs4 as dev dependency in a python:3.8.12 container with poetry@1.1.11 did not reproduce the issue. Ideally somehting I can reproduce inside a python container.

I’ll try to see if I can give you a minimal repro, but for me it started happening after the bs4 package was added and the cache from a previous github actions run was used.

Edit : it is run in a container based on ubuntu 20.04 where python is installed thanks to the setup python action. Poetry is installed with pip