setuptools: setuptools 45.0.0 may cause PyInstaller 3.3 packaged executable fail to launch

Observed the same behaviour on Python 3.7.5 with PyInstaller 3.5 and PyInstaller 3.6. I was able to build again successfully after reverting to previous release of setuptools:

pip install --upgrade 'setuptools<45.0.0'

Fixed by adding a hidden import:

a = Analysis(...,
             hiddenimports=['pkg_resources.py2_warn'],
             ...)

We ran into the same issue this morning, using PyInstaller 3.6

If you’re using pyinstaller from bash you can use “–hidden-import=‘pkg_resources.py2_warn’” to get rid of this issue. Eg: pyinstaller --hidden-import=‘pkg_resources.py2_warn’ ${package}

I am having the same issue with Python 3.6.8 and Pyinstaller 3.6. I suspect that it is caused by the recent commit pypa/setuptools@79f1694b05a66cc0fbbbf4e72d63d0a340cf6d84 in setuptools since it added the line __import__('pkg_resources.py2_warn') but not a regular import line like import pkg_resources.py2_warn (as done for other __import__ statements in pkg_resources.__init__.py) and afaik Pyinstaller does not scan for __import__ statements and hence does not freeze the file pkg_resources.py2_warn.py.

You either add this to your .spec-file if you use it before the Analysis step, or add it as a command-line parameter --hidden-import="pkg_resources.py2_warn"

Fixed by forcing in a hook file the following submodule collection:

...
hiddenimports.extend(collect_submodules('pkg_resources'))

My hack works whatever the version of setuptools. I’ll see if I have time to do a clean PR in here.

Shall we get a new update today or what is the plan?

Still an issue, got bitten by it with the latest Miniconda3. Fixed it on Windows by typing:

pyinstaller -F -c --hidden-import="pkg_resources.py2_warn" your_script.py

Note the use of double quotes, as single quotes do not work here on Windows.

Just occurred to me - why wait. So 49.1.1 is going out for release now.

In the commit (79f1694b05a66cc0fbbbf4e72d63d0a340cf6d84), a comment was made that this affects cxFreeze also.

The suggestion was made to simply use import to import the module, such as is done with py31compat. For py2warn, I chose to use __import__ to signal that the side effect is what’s desired (and to avoid linter warnings about an unused bound name, F401 imported but unused). With py31compat, the name is used later in the module; that’s not the case with py2warn.

As you can see, adjacent to this import, there are very similar import calls in this module that do not import any other names and there’s no comment or other tests preventing this sort of usage. I don’t understand why this import is causing trouble while others are not.

Probably tools like cxFreeze and pyInstaller should honor __import__ calls, especially when they’re at the top of an imported module. I don’t know why they’re not.

Even better, these tools should probably consider not relying primarily on runtime inference for module detection but instead rely on package metadata like importlib.metadata and Python Packaging tools like pip or pep517 to assemble these packages more deterministically.

Failing that, these tools should provide a mechanism for a project such as setuptools to advertise its “hidden” imports.

I’m sad to see there are so many workarounds for this issue but no robust solution. I’m happy to work with the maintainers of cxFreeze or PyInstaller to develop a robust solution.

Without a better recommendation, this behavior is working as intended, so I’m closing the ticket, but happy to re-open if there’s an actionable recommendation.

Python 3.7.8 Jun 28 2020, 08:53:46) [MSC v.1916 64 bit (AMD64)] on win32 I updated my python from 3.7.7 (32bit) to 3.7.8 (64bit). I started to get this mistake:

File “site-packages\pkg_resources_init_.py”, line 86, in <module> ModuleNotFoundError: No module named ‘pkg_resources.py2_warn’ [4188] Failed to execute script pyi_rth_pkgres

I was helped by a comment from the branch above: https://github.com/pypa/setuptools/issues/1963#issuecomment-574265532

my observation pyinstaller version=3.5 and setuptool 41.2.0 working correctly pynstaller.

我也遇到同樣的問題 對虛擬的環境重新更新pip 在C:\virenv\Scripts\ 目錄下 pip install --upgrade setuptools==44.0.0 就解決了

Is there a forum where maintainers of PyInstaller and cxFreeze (and others) convene to discuss designs around the common challenges of these tools? I have some ideas I’d be happy to share.

I’m not aware of any such thing and would guess that these designs just arose organically. I have also reflected that this puts all the work on the pyinstaller devs because you have to update the hooks in the pyinstaller repo, as opposed to updating them directly in each package according to some specified syntax as the package changes. And yeah, it misses imports that are not “standard”. It would be nice if there was a way for package maintainers to declare the various files that may need to be included in a app.

Maybe Python’s discourse would be a good place to bring up these ideas? I know pyinstaller is already struggling with maintainer time, so they may not be in a position to implement these ideas, but someone interested could?

But FYI, there is more complexity here than you may be considering. Bundling a fully functioning app should typically mean that you bundle the minimum modules required to run a package given the way the package is used by the app. E.g. consider a framework that provides some core modules, audio/video, and some other modules. Each of these modules depend on some external packages. E.g. audio/video may require a 100MB gstreamer package. Currently, pyinstaller should be able to tell which framework modules you use in the app and only bundle gstreamer if you actually use the audio/video module. I’m not sure pip style metadata would be able to do this?

Why is this closed if its still an issue?

The suggestion was made to simply use import to import the module, such as is done with py31compat. For py2warn, I chose to use __import__ to signal that the side effect is what’s desired (and to avoid linter warnings about an unused bound name, F401 imported but unused). With py31compat, the name is used later in the module; that’s not the case with py2warn.

As far as I understand (and this is not much), you could have reached the same effect with

import pkg_resources.py2_warn
del pkg_resources.py2_warn

This would at least have avoided the issue for cx_Freeze (beside being explicit). Anyhow, even if this would be an acceptable recommendation for you, it is “too late” in the sense that cx_Freeze probably needs to be patched to work with setuptools 45.0.0 anyway (see below).

As you can see, adjacent to this import, there are very similar import calls in this module that do not import any other names and there’s no comment or other tests preventing this sort of usage. I don’t understand why this import is causing trouble while others are not.

At least in cx_Freeze, there was already a hook for the other similar imports.

Failing that, these tools should provide a mechanism for a project such as setuptools to advertise its “hidden” imports.

The cx_Freeze mechanism is also to provide hooks to advertise the “hidden” import. I’ve opened a corresponding issue https://github.com/anthony-tuininga/cx_Freeze/issues/579 and made a PR proposal there.

Failing that, these tools should provide a mechanism for a project such as setuptools to advertise its “hidden” imports.

The pyinstaller mechanism is to use hooks that describe the hidden imports for each package. Looking at their hooks, it seems it should already be fixed in their master?

https://github.com/pyinstaller/pyinstaller/blob/develop/PyInstaller/hooks/hook-setuptools.py https://github.com/pyinstaller/pyinstaller/blob/develop/PyInstaller/hooks/hook-pkg_resources.py