poetry: InstalledRepository load() breaks if there's an empty string for a env.sys_path variable
- [x ] I am on the latest Poetry version.
- [x ] I have searched the issues of this repo and believe that this is not a duplicate.
- [x ] If an exception occurs when executing a command, I executed it again in debug mode (
-vvv
option).
- OS version and name: <Ubuntu 18.04.3>
- Poetry version: <1.0.0>
Issue
I ran into an error that pops up as: expected string or bytes-like object
. I followed the stack trace and found the location where it happens: /.poetry/lib/poetry/repositories/installed_repository.py
The problem is in the load() function where it calls to get a list of paths or something using the env.sys_path variable. For me it looked like this:
‘’’ for distribution in sorted( metadata.distributions(path=env.sys_path), key=lambda d: str(d._path), ): ‘’’
But I know that this is a little different between versions, so I’ll specifically say that the issue was the env.sys_path. My env.sys_path was returning one string that’s empty in it’s list. I don’t know why, I just know it does that. The empty string then is passed along with the rest of the code which assumes that it is a distribution package with metadata. It specifically passes along a None value which then breaks things later. The load() function needs to have logic to handle the case where env.sys_path has an empty string in the return values.
I was able to fix it by adding a simple check after the values are returned:
name = distribution.metadata["name"]
if name is None:
continue
I tried cleaning the names for env.sys_path before they’re used but then I get a different weird error:
[TypeError]
find_distributions() got an unexpected keyword argument 'name'
So I just did the simple fix where I check if name is None.
Simple fix, should be quick to add to the next release.
Stack trace:
[TypeError]
expected string or bytes-like object
Traceback (most recent call last):
File "/home/.poetry/lib/poetry/_vendor/py3.6/clikit/console_application.py", line 131, in run
status_code = command.handle(parsed_args, io)
File "/home/.poetry/lib/poetry/_vendor/py3.6/clikit/api/command/command.py", line 120, in handle
status_code = self._do_handle(args, io)
File "/home/.poetry/lib/poetry/_vendor/py3.6/clikit/api/command/command.py", line 171, in _do_handle
return getattr(handler, handler_method)(args, io, self)
File "/home/.poetry/lib/poetry/_vendor/py3.6/cleo/commands/command.py", line 92, in wrap_handle
return self.handle()
File "/home/.poetry/lib/poetry/console/commands/install.py", line 48, in handle
self.io, self.env, self.poetry.package, self.poetry.locker, self.poetry.pool
File "/home/.poetry/lib/poetry/installation/installer.py", line 55, in __init__
installed = self._get_installed()
File "/home/.poetry/lib/poetry/installation/installer.py", line 488, in _get_installed
return InstalledRepository.load(self._env)
File "/home/.poetry/lib/poetry/repositories/installed_repository.py", line 24, in load
package = Package(name, version, version)
File "/home/.poetry/lib/poetry/packages/package.py", line 42, in __init__
self._name = canonicalize_name(name)
File "/home/.poetry/lib/poetry/utils/helpers.py", line 31, in canonicalize_name
return _canonicalize_regex.sub('-', name).lower()
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 11
- Comments: 26 (3 by maintainers)
I’m experiencing the same issue when using
tox
.pip install tox
now installsvirtualenv==20.0.0
, but a few days ago16.7.9
was installed. Forcingvirtualenv<20
, i.e.pip install 'virtualenv<20' tox
, solves the problem for me, but it’s only a quick fix.A simple work-around is to change https://github.com/python-poetry/poetry/blob/master/poetry/repositories/installed_repository.py#L25 to:
To those still having problems even after upgrading virtualenv, see this comment (in short, remove any .egg-info folder from your project).
This issue is not caused by an empty string in
env.sys_path
, as the title says. (As mentioned above, the empty string is standard behaviour. Also, the exception is not raised on the code path involving the empty string, but when processing the site-packages directory.)Rather, the virtual environments created by virtualenv >= 20.0 contain marker files, which cause
importlib.metadata.distributions()
to report duplicate entries for packages, withNone
as their name and version.This has been reported to virtualenv here: https://github.com/pypa/virtualenv/issues/1589
This is resolved with virtualenv 20.0.3, released today.
@languitar Upgrade your CI to the latest virtualenv by removing the pin or pointing it to 20.0.3.
Any idea how to work around this issue? I am hit by this as well right now in a CI build but it doesn’t happen on my local machine with the same poetry version being used.
Thank you all for reporting and keeping the community up-to-date.
Good work 👍
The fix is in virtualenv’s master.
On a related note, Poetry depends on a rather old version of importlib_metadata (
~1.1.3
) for Python pre-3.8. Newer versions of this library are not affected by the bug. Upgrading to importlib_metadata 1.4 or 1.5 would prevent Poetry from being affected by this bug.Python 3.8.2 (due 2020-02-17) incorporates changes from importlib_metadata 1.4 and is therefore also not affected by the bug.