poetry: `poetry install` fails when installing dependencies, from git repositories, in gitlab CI.

I had a similar issue and it turns out the problem was that the library I was referring to (sentinelsat) had no master branch because they renamed their main branch to main. Since a lot of repositories have done that in the last few years, that might be the problem.

Adding the branch requirement in pyproject.toml worked for me:

sentinelsat = { git = "https://github.com/sentinelsat/sentinelsat.git", branch = "main" }

This problem already has an issue at #3366.

I believe this problem is related to something I’ve experienced.

It looks like poetry will clone the repository with it’s default name during poetry add but will try to infer the local repository name from the package name when doing anything else, so in my case I had something like:

poetry add git+ssh://git@git.mycompany.com:/mycompany-sdk.git

which cloned such repository under $HOME/.cache/pypoetry/virtualenvs/myproject-pbdKQ0bZ-py3.6/src/mycompany-sdk/

However, during a subsequent poetry update it failed with:

[CalledProcessError]
Command '['git', '--git-dir', '$HOME.cache/pypoetry/virtualenvs/myproject-pbdKQ0bZ-py3.6/src/mycompany/.git', '--work-tree', '$HOME/.cache/pypoetry/virtualenvs/myproject-pbdKQ0bZ-py3.6/src/mycompany', 'rev-parse', 'HEAD^{commit}']' returned non-zero exit status 128.

Which makes total sense because there’s no mycompany directory there, the repository is called mycompany-sdk, yet the package it contains is, indeed, named mycompany and that’s why poetry thinks it should look for that package there.

Experiencing exactly the same issue with 2 out of 4 github repositories. What the two that are failing have in common is that they both have submodules. Turns out I’d made a change to the way the submodules were being referenced.

I was able to identify the issue by manually adding git clone --recurse-submodules -- https://git@github.com/xxx/xxx.git /usr/local/src/xxx/xxx into the pipeline to see what the real error was. Turns out the submodules were using the url https://github.com, whereas their parents were accessed through https://git@github.com. I’d applied the following to ensure the PAT was being set for git@github.com;

git config --global url.https://$GITHUB_TOKEN@github.com/.insteadOf https://git@github.com/

This was not applying to the submodule and I got an authentication error when it tried to clone the submodule. I solved this by just updating the url to the submodules in .gitmodules.

This may not be the same issue as exit status code 1 is rather uninformative, but adding the command manually to your pipeline should reveal the true cause.

Experiencing this issue w/ a public GitHub repo.

Here is my Poetry config:

[package.source]
type = "git"
url = "ssh://git@github.com/agronholm/anyio.git"
reference = "4419414e25d17c456bdd3e5216f152dba27810de"
resolved_reference = "4419414e25d17c456bdd3e5216f152dba27810de"

And here at the GitLab CI Logs:

  CalledProcessError
  Command '['git', 'clone', '--recurse-submodules', '--', 'ssh://git@github.com/agronholm/anyio.git', '/usr/local/src/anyio']' returned non-zero exit status 128.
  at /usr/local/lib/python3.9/site-packages/poetry/utils/_compat.py:217 in run
      213│                 process.wait()
      214│                 raise
      215│             retcode = process.poll()
      216│             if check and retcode:
    → 217│                 raise CalledProcessError(
      218│                     retcode, process.args, output=stdout, stderr=stderr
      219│                 )
      220│         finally:
      221│             # None because our context manager __exit__ does not use them.
The command '/bin/sh -c poetry install --no-dev' returned a non-zero code: 1
Cleaning up file based variables
00:00
ERROR: Job failed: command terminated with exit code 1

Anyone aware of a workaround?

I just add #main at the end. It’s worked. 😄

I had a similar issue with a dependency from a private Gitlab repository that used SSH for cloning (same CalledProcessError). When testing in a local Docker container, I noticed that I had to actively confirm the server fingerprint. As git has no option for automatically doing that, I solved poetry’s installation of dependencies in the Gitlab CI container by adding the fingerprint to ~/.ssh/known_hosts before running poetry install in my .gitlab-ci.yml. Here’s how I did that:

1. Manually clone my dependency’s repository in a clean docker container
2. In that docker container, get the two lines (not sure why there’s more than one) from ~/.ssh/known_hosts
3. In my .gitlab-ci.yml, add the following lines before calling poetry install:

- mkdir ~/.ssh
- echo "FIRST LINE COPIED IN STEP 2" >> ~/.ssh/known_hosts
- echo "SECOND LINE COPIED IN STEP 2" >> ~/.ssh/known_hosts 

I’m aware that the problem in the original comment might be something else (given that you, @pawelrubin, were trying to clone via HTTPS from a public repository), but maybe this will help other people ending up here when researching that error message.

https://github.com/python-poetry/poetry/pull/5428 was merged against master branch, which corresponds to version 1.2 of Poetry, which is still in beta.

Could you try on latest beta?

curl -sSL https://install.python-poetry.org | python3 - --preview

Otherwise, for 1.1, https://github.com/python-poetry/poetry/issues/2475#issuecomment-1061587214 may also fix the issue, as mentioned.

In case it helps anyone: I had the same “returned non-zero exit status 128” issue. I investigated it by just running this:

git clone --recurse-submodules -- SOME_REPO /usr/local/src/PACKAGE_NAME

It turned out that /usr/local/src did not exist / wasn’t writable, so totally a problem on my end. It would really be nice if Poetry could log the stderr of Git to make debugging situations like this easier.

For those wondering: The path is coming from the pip_installer which uses sys.prefix / "src"

In case it helps anyone: I had the same “returned non-zero exit status 128” issue. I investigated it by just running this:

git clone --recurse-submodules -- SOME_REPO /usr/local/src/PACKAGE_NAME

It turned out that /usr/local/src did not exist / wasn’t writable, so totally a problem on my end. It would really be nice if Poetry could log the stderr of Git to make debugging situations like this easier.

For those wondering: The path is coming from the pip_installer which uses sys.prefix / "src"

Thx so much for your help. Same problem here. But we’ve fixed it by setting: poetry config virtualenvs.create true && poetry config virtualenvs.in-project true In my understanding the clone then will put the repo content inside the virtualenv instead of trying to place it into /usr/…

Tried upgrading to the preview version 1.2.0b3 and now getting another error. Neither the 1.1 comment nor upgrading has worked for poetry add git+https://github.com/commonknowledge/wagtail-localize.git#main

Command ['/usr/local/bin/python3.9', '/usr/local/venv/lib/python3.9/site-packages/virtualenv/seed/wheels/embed/pip-22.2.2-py3-none-any.whl/pip', 'install', '--disable-pip-version-check', '--prefix', '/usr/local', '--no-deps', '/home/vscode/.cache/pypoetry/artifacts/08/9f/47/c16ae03035fc69eaf100ea39657a49baaeef714e25a52575710c34cd48/six-1.16.0-py2.py3-none-any.whl'] errored with the following
      1459│ 
  • Installing six (1.16.0): Failed

  CalledProcessError

  Command '['/usr/local/bin/python3.9', '/usr/local/venv/lib/python3.9/site-packages/virtualenv/seed/wheels/embed/pip-22.2.2-py3-none-any.whl/pip', 'install', '--disable-pip-version-check', '--prefix', '/usr/local', '--no-deps', '/home/vscode/.cache/pypoetry/artifacts/08/9f/47/c16ae03035fc69eaf100ea39657a49baaeef714e25a52575710c34cd48/six-1.16.0-py2.py3-none-any.whl']' returned non-zero exit status 1.

  at /usr/local/lib/python3.9/subprocess.py:528 in run
       524│             # We don't call process.wait() as .__exit__ does that for us.
       525│             raise
       526│         retcode = process.poll()
       527│         if check and retcode:
    →  528│             raise CalledProcessError(retcode, process.args,
       529│                                      output=stdout, stderr=stderr)
       530│     return CompletedProcess(process.args, retcode, stdout, stderr)
       531│ 
       532│ 

The following error occurred when trying to handle this error:


  EnvCommandError

  Command ['/usr/local/bin/python3.9'