numcodecs: Pyinstaller failing on numcodecs blosc import

Minimal, reproducible code sample, a copy-pastable example if possible

from aicsimageio import AICSImage


then

pyinstaller -F -w main.py

Problem description

I’m using aicsimageio for loading CZI files and I’m trying to package it up into a stand-alone app for one of our EngD students to use.

The only reference I could find to my exact problem is this: https://github.com/napari/napari/issues/665#issuecomment-551965687

And I couldn’t find the ticket. So I thought I would raise it.

The issue is that the app works when run as a script, the app functions completely as expected. When I package it up I get the following error:

Traceback (most recent call last): File “main.py”, line 3, in <module> File “PyInstaller\loader\pyimod03_importers.py”, line 476, in exec_module File “control_scheme_python.py”, line 11, in <module> File “PyInstaller\loader\pyimod03_importers.py”, line 476, in exec_module File “chunking_logic.py”, line 6, in <module> File “PyInstaller\loader\pyimod03_importers.py”, line 476, in exec_module File “aicsimageio_init_.py”, line 6, in <module> File “PyInstaller\loader\pyimod03_importers.py”, line 476, in exec_module File “aicsimageio\aics_image.py”, line 10, in <module> File “PyInstaller\loader\pyimod03_importers.py”, line 476, in exec_module File “xarray_init_.py”, line 1, in <module> File “PyInstaller\loader\pyimod03_importers.py”, line 476, in exec_module File “xarray\tutorial.py”, line 13, in <module> File “PyInstaller\loader\pyimod03_importers.py”, line 476, in exec_module File “xarray\backends_init_.py”, line 17, in <module> File “PyInstaller\loader\pyimod03_importers.py”, line 476, in exec_module File “xarray\backends\zarr.py”, line 22, in <module> File “PyInstaller\loader\pyimod03_importers.py”, line 476, in exec_module File “zarr_init_.py”, line 2, in <module> File “PyInstaller\loader\pyimod03_importers.py”, line 476, in exec_module File “zarr\codecs.py”, line 3, in <module> ImportError: cannot import name ‘Blosc’ from ‘numcodecs’ (C:\Users\iris\AppData\Local\Temp_MEI63202\numcodecs_init_.pyc)

I guess the expected behavior is: it runs.

I had a c++ installer (MS VS build tools 2019) and the build from pip for aicsimageio ran without error - including seeing the numcodecs build.

The most desirable and quickest solution I’m hoping for is that someone points out a stupid mistake I’ve made and I fix it easily 😃

Version and installation information

Please provide the following:

  • Value of numcodecs.__version__ = 0.9.1
  • Python 3.9.7
  • Win 10 19042.1348
  • using pip into a clean VM spun up exclusively for this bug_requirements.txt

I’ve had it on other versions of python and conda too.

Thanks

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 30 (15 by maintainers)

Commits related to this issue

Most upvoted comments

Interesting. Good to know that compat_ext is the module with issues. No other modules had issues?

FWIW compat_ext is the only one that supplies a .pxd file. Not sure if that is relevant. Maybe worth digging into further.

PyInstaller includes a number of hooks itself. So another option may be for someone interested in this to add the hook upstream.

I tried having a single hidden import of numcodecs.compat_ext, and it seemed to work. I guess that is the only module that pyinstaller misses, and it is missed because it is only imported in pyx files.

I added the hook as a PR, which should fix this issue: pyinstaller/pyinstaller-hooks-contrib#420

That pyinstaller-hooks-contrib package is automatically installed as well when pyinstaller is installed.

I’m not familiar with how numcodecs works, but PyInstaller should pack the entire thing if using the hook method above. Looking at the package I don’t see anything within numcodecs that would obviously cause problems.

Missing imports are a common theme with PyInstaller, and might be expected in situations where your program could need an ambiguous codec to read a file but it’s never explicitly imported within your code. Good luck!

Thanks @psavery! 😄

Hi all. https://github.com/pyinstaller/pyinstaller-hooks-contrib/pull/420 was merged. I gave it a try on my local example, and it seems to have fixed the issue!

Until a new release comes out, I guess we can install from master via a command like pip install git+https://github.com/pyinstaller/pyinstaller-hooks-contrib.