google-cloud-python: pyinstaller executable can't find package info

The pyinstaller complains the error with the following line:

from gcloud.pubsub.client import Client

The module is installed correctly in /usr/local/lib/python2.7/dist-packages. I run setup.py (build/ install) in addition pip install --upgrade cloud too.

Traceback (most recent call last):
  File "<string>", line 267, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 271, in load_module
    exec(bytecode, module.__dict__)
  File "/home/mtwu/gce/cloudn/cloudx-local/gce/gce_pubsub.py", line 6, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 271, in load_module
    exec(bytecode, module.__dict__)
  File "/home/mtwu/gce/cloudn/install/build/cloudx_cli/out00-PYZ.pyz/gcloud", line 19, in <module>
  File "/home/mtwu/gce/cloudn/install/build/cloudx_cli/out00-PYZ.pyz/pkg_resources", line 551, in get_distribution
The following import will cause the following pyinstaller error.
from gcloud.pubsub.client import Client
  File "/home/mtwu/gce/cloudn/install/build/cloudx_cli/out00-PYZ.pyz/pkg_resources", line 431, in get_provider
  File "/home/mtwu/gce/cloudn/install/build/cloudx_cli/out00-PYZ.pyz/pkg_resources", line 952, in require
  File "/home/mtwu/gce/cloudn/install/build/cloudx_cli/out00-PYZ.pyz/pkg_resources", line 839, in resolve
pkg_resources.DistributionNotFound: The 'gcloud' distribution was not found and is required by the application

mtwu@ubuntu:/usr/local/lib/python2.7/dist-packages$ ls -l gcloud 
total 360
drwxr-sr-x 2 root staff  4096 Oct 16 09:13 bigquery
drwxr-sr-x 3 root staff  4096 Oct 16 09:13 bigtable
-rw-r--r-- 1 root staff  7564 Oct 16 09:13 client.py
-rw-r--r-- 1 root staff  8038 Oct 16 09:13 client.pyc
-rw-r--r-- 1 root staff 12949 Oct 16 09:13 connection.py
-rw-r--r-- 1 root staff 12304 Oct 16 09:13 connection.pyc
-rw-r--r-- 1 root staff 14075 Oct 16 09:13 credentials.py
-rw-r--r-- 1 root staff 13661 Oct 16 09:13 credentials.pyc
drwxr-sr-x 3 root staff  4096 Oct 16 09:13 datastore
-rw-r--r-- 1 root staff  4036 Oct 16 09:13 demo.py
-rw-r--r-- 1 root staff  4010 Oct 16 09:13 demo.pyc
drwxr-sr-x 2 root staff  4096 Oct 16 09:13 dns
-rw-r--r-- 1 root staff  1418 Oct 16 09:13 environment_vars.py
-rw-r--r-- 1 root staff   647 Oct 16 09:13 environment_vars.pyc
-rw-r--r-- 1 root staff  5819 Oct 16 09:13 exceptions.py
-rw-r--r-- 1 root staff  9026 Oct 16 09:13 exceptions.pyc
-rw-r--r-- 1 root staff  8048 Oct 16 09:13 _helpers.py
-rw-r--r-- 1 root staff 10032 Oct 16 09:13 _helpers.pyc
-rw-r--r-- 1 root staff   736 Oct 16 09:13 __init__.py
-rw-r--r-- 1 root staff   333 Oct 16 09:13 __init__.pyc
-rw-r--r-- 1 root staff  4957 Oct 16 09:13 iterator.py
-rw-r--r-- 1 root staff  5210 Oct 16 09:13 iterator.pyc
drwxr-sr-x 2 root staff  4096 Oct 16 09:13 pubsub
drwxr-sr-x 2 root staff  4096 Oct 16 09:13 resource_manager
drwxr-sr-x 2 root staff  4096 Oct 16 09:13 search
drwxr-sr-x 3 root staff  4096 Oct 16 09:13 storage
-rw-r--r-- 1 root staff  7021 Oct 16 09:13 test_client.py
-rw-r--r-- 1 root staff  9101 Oct 16 09:13 test_client.pyc
-rw-r--r-- 1 root staff 13090 Oct 16 09:13 test_connection.py
-rw-r--r-- 1 root staff 15451 Oct 16 09:13 test_connection.pyc
-rw-r--r-- 1 root staff 22622 Oct 16 09:13 test_credentials.py
-rw-r--r-- 1 root staff 28116 Oct 16 09:13 test_credentials.pyc
-rw-r--r-- 1 root staff  2844 Oct 16 09:13 test_exceptions.py
-rw-r--r-- 1 root staff  3664 Oct 16 09:13 test_exceptions.pyc
-rw-r--r-- 1 root staff 12177 Oct 16 09:13 test__helpers.py
-rw-r--r-- 1 root staff 19874 Oct 16 09:13 test__helpers.pyc
-rw-r--r-- 1 root staff  1143 Oct 16 09:13 _testing.py
-rw-r--r-- 1 root staff  1212 Oct 16 09:13 _testing.pyc
-rw-r--r-- 1 root staff  6781 Oct 16 09:13 test_iterator.py
-rw-r--r-- 1 root staff  8076 Oct 16 09:13 test_iterator.pyc

About this issue

Original URL
State: closed
Created 9 years ago
Reactions: 1
Comments: 31 (9 by maintainers)

Commits related to this issue

PyInstaller: add hooks for google.cloud and google.cloud.storage Fixes https://github.com/GoogleCloudPlatform/google-cloud-python/issues/1187 — committed to efiop/pyinstaller by efiop 7 years ago
PyInstaller: add hooks for google.cloud and google.cloud.storage, v2 Fixes https://github.com/GoogleCloudPlatform/google-cloud-python/issues/1187 v2: added copyright headers — committed to efiop/pyinstaller by efiop 7 years ago
PyInstaller: add hooks for google.cloud and google.cloud.storage, v3 Fixes https://github.com/GoogleCloudPlatform/google-cloud-python/issues/1187 v2: added copyright headers v3: fixed year — committed to efiop/pyinstaller by efiop 7 years ago

Most upvoted comments

You will need to create a pyinstaller hook for packages that depend on setuptools’ get_distribution() to obtain metadata information about the package.

Create a hook file in some hooks directory. The file name should be hook-<modulename>.py for pyinstaller to find it properly (i.e. hook-gcloud.py).

The content will be:

from PyInstaller.utils.hooks import copy_metadata
datas = copy_metadata('google-cloud-core')

The hooks directory should be set in the Analysis phase of your pyinstaller .spec file:

a = Analysis(['bogus.py'],
             pathex=[],
             binaries=[],
             hiddenimports=[],
             hookspath="/path/to/your/hooks/dir",
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher)

I haven’t tested the above, but this is what worked for me for another python package which is doing something similar (APScheduler).

If and when you get this working, consider contributing the hook back to pyinstaller project.

+16

jessesuen on Oct 18, 2016

In the root folder of bogus.py two directories (build, dist) are created after executing

$ pyinstaller bogus.py

execute the binary by

$ cd /dist/bogus && ./bogus

and the said error should pop up.

I updated to google-cloud as instructed, the problem persists:

  Traceback (most recent call last):
  File "bogus.py", line 1, in <module>
  File "/tmp/pip-build-JhwLSY/pyinstaller/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module
  File "google/cloud/datastore/__init__.py", line 55, in <module>
  File "/tmp/pip-build-JhwLSY/pyinstaller/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module
  File "google/cloud/datastore/connection.py", line 23, in <module>
  File "/tmp/pip-build-JhwLSY/pyinstaller/PyInstaller/loader/pyimod03_importers.py", line 389, in load_module
  File "google/cloud/connection.py", line 31, in <module>
  File "pkg_resources/__init__.py", line 559, in get_distribution
  File "pkg_resources/__init__.py", line 433, in get_provider
  File "pkg_resources/__init__.py", line 970, in require
  File "pkg_resources/__init__.py", line 856, in resolve
pkg_resources.DistributionNotFound: The 'google-cloud-core' distribution was not found and is required by the application
Failed to execute script bogus

As @mtwu-aviatrix commented on Oct 29, 2015, the native interpreter has no problem with this import. The problem arises when it has been bundled by Pyinstaller (ver. 3.2).

hu7241 on Oct 12, 2016