cloudmapper: Cloudmapper fails to install on macos

Hello, whatever the solution I came up with to setup cloudmapper, it fails at the pyjq install…

pip3 install --user pipenv
cd cloudmapper/; pipenv install --skip-lock

is it related to the below issue ?

https://github.com/doloopwhile/pyjq/issues/54 https://bugs.python.org/issue42504

Please find hereunder my logs:

Installing dependencies from Pipfile… An error occurred while installing pyjq==2.3.1! Will try again. An error occurred while installing seaborn==0.10.1! Will try again. 🐍 β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰ 37/37 β€” 00:00:44 Installing initially failed dependencies… [InstallError]: File β€œ/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/cli/command.py”, line 233, in install [InstallError]: retcode = do_install( [InstallError]: File β€œ/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/core.py”, line 2052, in do_install [InstallError]: do_init( [InstallError]: File β€œ/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/core.py”, line 1304, in do_init [InstallError]: do_install_dependencies( [InstallError]: File β€œ/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/core.py”, line 899, in do_install_dependencies [InstallError]: batch_install( [InstallError]: File β€œ/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/core.py”, line 796, in batch_install [InstallError]: _cleanup_procs(procs, failed_deps_queue, retry=retry) [InstallError]: File β€œ/usr/local/Cellar/pipenv/2020.11.15/libexec/lib/python3.9/site-packages/pipenv/core.py”, line 703, in _cleanup_procs [InstallError]: raise exceptions.InstallError(c.dep.name, extra=err_lines) [pipenv.exceptions.InstallError]: Collecting pyjq==2.3.1 [pipenv.exceptions.InstallError]: Using cached pyjq-2.3.1.tar.gz (2.1 MB) [pipenv.exceptions.InstallError]: Requirement already satisfied: six in /Users/hiyaa/.local/share/virtualenvs/cloudmapper-WzNltrOW/lib/python3.8/site-packages (from pyjq==2.3.1->-r /var/folders/26/pmmcdlhx727ffbxy2ld7m86h0000gn/T/pipenv-8bez_ooa-requirements/pipenv-r8czubvu-requirement.txt (line 1)) (1.14.0) [pipenv.exceptions.InstallError]: Building wheels for collected packages: pyjq [pipenv.exceptions.InstallError]: Building wheel for pyjq (setup.py): started [pipenv.exceptions.InstallError]: Building wheel for pyjq (setup.py): finished with status β€˜error’ [pipenv.exceptions.InstallError]: ERROR: Command errored out with exit status 1: [pipenv.exceptions.InstallError]: command: /Users/hiyaa/.local/share/virtualenvs/cloudmapper-WzNltrOW/bin/python -u -c β€˜import sys, setuptools, tokenize; sys.argv[0] = β€˜"’"’/private/var/folders/26/pmmcdlhx727ffbxy2ld7m86h0000gn/T/pip-install-7m4e96ks/pyjq_abce72f03b2c422d9a98ef71102e0034/setup.pyβ€™β€œ'β€β€˜; file=β€™β€œ'β€β€˜/private/var/folders/26/pmmcdlhx727ffbxy2ld7m86h0000gn/T/pip-install-7m4e96ks/pyjq_abce72f03b2c422d9a98ef71102e0034/setup.pyβ€™β€œ'β€β€˜;f=getattr(tokenize, β€˜"β€™β€œβ€˜openβ€™β€β€™β€œβ€˜, open)(file);code=f.read().replace(β€™β€β€˜"’\r\nβ€™β€œ'β€β€˜, β€˜"’"’\nβ€™β€œ'β€β€˜);f.close();exec(compile(code, file, β€˜"β€™β€œβ€˜execβ€™β€β€™β€œβ€˜))’ bdist_wheel -d /private/var/folders/26/pmmcdlhx727ffbxy2ld7m86h0000gn/T/pip-wheel-dbvkaevy [pipenv.exceptions.InstallError]: cwd: /private/var/folders/26/pmmcdlhx727ffbxy2ld7m86h0000gn/T/pip-install-7m4e96ks/pyjq_abce72f03b2c422d9a98ef71102e0034/ [pipenv.exceptions.InstallError]: Complete output (44 lines): [pipenv.exceptions.InstallError]: running bdist_wheel [pipenv.exceptions.InstallError]: running build [pipenv.exceptions.InstallError]: running build_py [pipenv.exceptions.InstallError]: creating build [pipenv.exceptions.InstallError]: creating build/lib.macosx-11-x86_64-3.8 [pipenv.exceptions.InstallError]: copying pyjq.py -> build/lib.macosx-11-x86_64-3.8 [pipenv.exceptions.InstallError]: warning: build_py: byte-compiling is disabled, skipping. [pipenv.exceptions.InstallError]: [pipenv.exceptions.InstallError]: running build_ext [pipenv.exceptions.InstallError]: Traceback (most recent call last): [pipenv.exceptions.InstallError]: File β€œ<string>”, line 1, in <module> [pipenv.exceptions.InstallError]: File β€œ/private/var/folders/26/pmmcdlhx727ffbxy2ld7m86h0000gn/T/pip-install-7m4e96ks/pyjq_abce72f03b2c422d9a98ef71102e0034/setup.py”, line 93, in <module> [pipenv.exceptions.InstallError]: setup( [pipenv.exceptions.InstallError]: File β€œ/Users/hiyaa/.local/share/virtualenvs/cloudmapper-WzNltrOW/lib/python3.8/site-packages/setuptools/init.py”, line 153, in setup [pipenv.exceptions.InstallError]: return distutils.core.setup(**attrs) [pipenv.exceptions.InstallError]: File β€œ/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/core.py”, line 148, in setup [pipenv.exceptions.InstallError]: dist.run_commands() [pipenv.exceptions.InstallError]: File β€œ/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/dist.py”, line 966, in run_commands [pipenv.exceptions.InstallError]: self.run_command(cmd) [pipenv.exceptions.InstallError]: File β€œ/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/dist.py”, line 985, in run_command [pipenv.exceptions.InstallError]: cmd_obj.run() [pipenv.exceptions.InstallError]: File β€œ/Users/hiyaa/.local/share/virtualenvs/cloudmapper-WzNltrOW/lib/python3.8/site-packages/wheel/bdist_wheel.py”, line 299, in run [pipenv.exceptions.InstallError]: self.run_command(β€˜build’) [pipenv.exceptions.InstallError]: File β€œ/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/cmd.py”, line 313, in run_command [pipenv.exceptions.InstallError]: self.distribution.run_command(command) [pipenv.exceptions.InstallError]: File β€œ/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/dist.py”, line 985, in run_command [pipenv.exceptions.InstallError]: cmd_obj.run() [pipenv.exceptions.InstallError]: File β€œ/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/command/build.py”, line 135, in run [pipenv.exceptions.InstallError]: self.run_command(cmd_name) [pipenv.exceptions.InstallError]: File β€œ/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/cmd.py”, line 313, in run_command [pipenv.exceptions.InstallError]: self.distribution.run_command(command) [pipenv.exceptions.InstallError]: File β€œ/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/dist.py”, line 985, in run_command [pipenv.exceptions.InstallError]: cmd_obj.run() [pipenv.exceptions.InstallError]: File β€œ/private/var/folders/26/pmmcdlhx727ffbxy2ld7m86h0000gn/T/pip-install-7m4e96ks/pyjq_abce72f03b2c422d9a98ef71102e0034/setup.py”, line 30, in run [pipenv.exceptions.InstallError]: self._build_oniguruma() [pipenv.exceptions.InstallError]: File β€œ/private/var/folders/26/pmmcdlhx727ffbxy2ld7m86h0000gn/T/pip-install-7m4e96ks/pyjq_abce72f03b2c422d9a98ef71102e0034/setup.py”, line 39, in _build_oniguruma [pipenv.exceptions.InstallError]: self._build_lib( [pipenv.exceptions.InstallError]: File β€œ/private/var/folders/26/pmmcdlhx727ffbxy2ld7m86h0000gn/T/pip-install-7m4e96ks/pyjq_abce72f03b2c422d9a98ef71102e0034/setup.py”, line 67, in _build_lib [pipenv.exceptions.InstallError]: os.environ[β€˜MACOSX_DEPLOYMENT_TARGET’] = macosx_deployment_target [pipenv.exceptions.InstallError]: File β€œ/usr/local/opt/python@3.8/bin/…/Frameworks/Python.framework/Versions/3.8/lib/python3.8/os.py”, line 680, in setitem [pipenv.exceptions.InstallError]: value = self.encodevalue(value) [pipenv.exceptions.InstallError]: File β€œ/usr/local/opt/python@3.8/bin/…/Frameworks/Python.framework/Versions/3.8/lib/python3.8/os.py”, line 750, in encode [pipenv.exceptions.InstallError]: raise TypeError(β€œstr expected, not %s” % type(value).name) [pipenv.exceptions.InstallError]: TypeError: str expected, not int [pipenv.exceptions.InstallError]: ---------------------------------------- [pipenv.exceptions.InstallError]: ERROR: Failed building wheel for pyjq [pipenv.exceptions.InstallError]: Running setup.py clean for pyjq [pipenv.exceptions.InstallError]: Failed to build pyjq [pipenv.exceptions.InstallError]: Installing collected packages: pyjq [pipenv.exceptions.InstallError]: Running setup.py install for pyjq: started [pipenv.exceptions.InstallError]: Running setup.py install for pyjq: finished with status β€˜error’ [pipenv.exceptions.InstallError]: ERROR: Command errored out with exit status 1: [pipenv.exceptions.InstallError]: command: /Users/hiyaa/.local/share/virtualenvs/cloudmapper-WzNltrOW/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = 'β€β€˜"’/private/var/folders/26/pmmcdlhx727ffbxy2ld7m86h0000gn/T/pip-install-7m4e96ks/pyjq_abce72f03b2c422d9a98ef71102e0034/setup.pyβ€™β€œ'β€β€˜; file=β€™β€œ'β€β€˜/private/var/folders/26/pmmcdlhx727ffbxy2ld7m86h0000gn/T/pip-install-7m4e96ks/pyjq_abce72f03b2c422d9a98ef71102e0034/setup.pyβ€™β€œ'β€β€˜;f=getattr(tokenize, β€˜"β€™β€œβ€˜openβ€™β€β€™β€œβ€˜, open)(file);code=f.read().replace(β€™β€β€˜"’\r\nβ€™β€œ'β€β€˜, β€˜"’"’\nβ€™β€œ'β€β€˜);f.close();exec(compile(code, file, β€˜"β€™β€œβ€˜execβ€™β€β€™β€œβ€˜))’ install --record /private/var/folders/26/pmmcdlhx727ffbxy2ld7m86h0000gn/T/pip-record-1eezjp50/install-record.txt --single-version-externally-managed --compile --install-headers /Users/hiyaa/.local/share/virtualenvs/cloudmapper-WzNltrOW/include/site/python3.8/pyjq [pipenv.exceptions.InstallError]: cwd: /private/var/folders/26/pmmcdlhx727ffbxy2ld7m86h0000gn/T/pip-install-7m4e96ks/pyjq_abce72f03b2c422d9a98ef71102e0034/ [pipenv.exceptions.InstallError]: Complete output (46 lines): [pipenv.exceptions.InstallError]: running install [pipenv.exceptions.InstallError]: running build [pipenv.exceptions.InstallError]: running build_py [pipenv.exceptions.InstallError]: creating build [pipenv.exceptions.InstallError]: creating build/lib.macosx-11-x86_64-3.8 [pipenv.exceptions.InstallError]: copying pyjq.py -> build/lib.macosx-11-x86_64-3.8 [pipenv.exceptions.InstallError]: warning: build_py: byte-compiling is disabled, skipping. [pipenv.exceptions.InstallError]: [pipenv.exceptions.InstallError]: running build_ext [pipenv.exceptions.InstallError]: Traceback (most recent call last): [pipenv.exceptions.InstallError]: File β€œ<string>”, line 1, in <module> [pipenv.exceptions.InstallError]: File β€œ/private/var/folders/26/pmmcdlhx727ffbxy2ld7m86h0000gn/T/pip-install-7m4e96ks/pyjq_abce72f03b2c422d9a98ef71102e0034/setup.py”, line 93, in <module> [pipenv.exceptions.InstallError]: setup( [pipenv.exceptions.InstallError]: File β€œ/Users/hiyaa/.local/share/virtualenvs/cloudmapper-WzNltrOW/lib/python3.8/site-packages/setuptools/init.py”, line 153, in setup [pipenv.exceptions.InstallError]: return distutils.core.setup(**attrs) [pipenv.exceptions.InstallError]: File β€œ/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/core.py”, line 148, in setup [pipenv.exceptions.InstallError]: dist.run_commands() [pipenv.exceptions.InstallError]: File β€œ/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/dist.py”, line 966, in run_commands [pipenv.exceptions.InstallError]: self.run_command(cmd) [pipenv.exceptions.InstallError]: File β€œ/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/dist.py”, line 985, in run_command [pipenv.exceptions.InstallError]: cmd_obj.run() [pipenv.exceptions.InstallError]: File β€œ/Users/hiyaa/.local/share/virtualenvs/cloudmapper-WzNltrOW/lib/python3.8/site-packages/setuptools/command/install.py”, line 61, in run [pipenv.exceptions.InstallError]: return orig.install.run(self) [pipenv.exceptions.InstallError]: File β€œ/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/command/install.py”, line 545, in run [pipenv.exceptions.InstallError]: self.run_command(β€˜build’) [pipenv.exceptions.InstallError]: File β€œ/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/cmd.py”, line 313, in run_command [pipenv.exceptions.InstallError]: self.distribution.run_command(command) [pipenv.exceptions.InstallError]: File β€œ/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/dist.py”, line 985, in run_command [pipenv.exceptions.InstallError]: cmd_obj.run() [pipenv.exceptions.InstallError]: File β€œ/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/command/build.py”, line 135, in run [pipenv.exceptions.InstallError]: self.run_command(cmd_name) [pipenv.exceptions.InstallError]: File β€œ/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/cmd.py”, line 313, in run_command [pipenv.exceptions.InstallError]: self.distribution.run_command(command) [pipenv.exceptions.InstallError]: File β€œ/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/distutils/dist.py”, line 985, in run_command [pipenv.exceptions.InstallError]: cmd_obj.run() [pipenv.exceptions.InstallError]: File β€œ/private/var/folders/26/pmmcdlhx727ffbxy2ld7m86h0000gn/T/pip-install-7m4e96ks/pyjq_abce72f03b2c422d9a98ef71102e0034/setup.py”, line 30, in run [pipenv.exceptions.InstallError]: self._build_oniguruma() [pipenv.exceptions.InstallError]: File β€œ/private/var/folders/26/pmmcdlhx727ffbxy2ld7m86h0000gn/T/pip-install-7m4e96ks/pyjq_abce72f03b2c422d9a98ef71102e0034/setup.py”, line 39, in _build_oniguruma [pipenv.exceptions.InstallError]: self._build_lib( [pipenv.exceptions.InstallError]: File β€œ/private/var/folders/26/pmmcdlhx727ffbxy2ld7m86h0000gn/T/pip-install-7m4e96ks/pyjq_abce72f03b2c422d9a98ef71102e0034/setup.py”, line 67, in _build_lib [pipenv.exceptions.InstallError]: os.environ[β€˜MACOSX_DEPLOYMENT_TARGET’] = macosx_deployment_target [pipenv.exceptions.InstallError]: File β€œ/usr/local/opt/python@3.8/bin/…/Frameworks/Python.framework/Versions/3.8/lib/python3.8/os.py”, line 680, in setitem [pipenv.exceptions.InstallError]: value = self.encodevalue(value) [pipenv.exceptions.InstallError]: File β€œ/usr/local/opt/python@3.8/bin/…/Frameworks/Python.framework/Versions/3.8/lib/python3.8/os.py”, line 750, in encode [pipenv.exceptions.InstallError]: raise TypeError(β€œstr expected, not %s” % type(value).name) [pipenv.exceptions.InstallError]: TypeError: str expected, not int [pipenv.exceptions.InstallError]: ---------------------------------------- [pipenv.exceptions.InstallError]: ERROR: Command errored out with exit status 1: /Users/hiyaa/.local/share/virtualenvs/cloudmapper-WzNltrOW/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = 'β€β€˜"’/private/var/folders/26/pmmcdlhx727ffbxy2ld7m86h0000gn/T/pip-install-7m4e96ks/pyjq_abce72f03b2c422d9a98ef71102e0034/setup.pyβ€™β€œ'β€β€˜; file=β€™β€œ'β€β€˜/private/var/folders/26/pmmcdlhx727ffbxy2ld7m86h0000gn/T/pip-install-7m4e96ks/pyjq_abce72f03b2c422d9a98ef71102e0034/setup.pyβ€™β€œ'β€β€˜;f=getattr(tokenize, β€˜"β€™β€œβ€˜openβ€™β€β€™β€œβ€˜, open)(file);code=f.read().replace(β€™β€β€˜"’\r\nβ€™β€œ'β€β€˜, β€˜"’"’\nβ€™β€œ'β€β€˜);f.close();exec(compile(code, file, β€˜"β€™β€œβ€˜exec’”’"β€˜))’ install --record /private/var/folders/26/pmmcdlhx727ffbxy2ld7m86h0000gn/T/pip-record-1eezjp50/install-record.txt --single-version-externally-managed --compile --install-headers /Users/hiyaa/.local/share/virtualenvs/cloudmapper-WzNltrOW/include/site/python3.8/pyjq Check the logs for full command output. ERROR: Couldn’t install package: pyjq Package installation failed… ☀ β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰β–‰ 1/2 β€”

Many thanks !

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 1
  • Comments: 18 (8 by maintainers)

Most upvoted comments

You’re not wrong @w0rmr1d3r, I did manually implement #876 (which I see you reviewed a couple of weeks before I opened my issue), although it was running sudo apt install autoconf libtool from doloopwhile/pyjq#50 which finally fixed the issue for me.

Ideally I would reset my environment to test how much of an impact manually implementing #876 has before installed the additional apt packages.

That happens to you the same way it happened to other people, because urllib3 got updated and it conflicted with botocore versions and other dependencies. Exists this PR which fixes the issue that you have encountered -> https://github.com/duo-labs/cloudmapper/pull/876 Which, at the same time, it fixes this issue -> https://github.com/duo-labs/cloudmapper/issues/875

That’s why I was saying that there are issues and PRs opened regarding dependencies and problems installing them.

issues fixed by updating typed-ast==1.5.5 and pyjq==2.6.0

@w0rmr1d3r correct, my file looks like this: astroid autoflake autopep8 boto3 botocore certifi chardet coverage docutils idna isort Jinja2 jmespath lazy-object-proxy MarkupSafe matplotlib mccabe mock netaddr nose pandas parliament policyuniverse pycodestyle pyflakes pyjq pylint python-dateutil PyYAML requests s3transfer seaborn six toml typed-ast urllib3 wrapt

I have to say, I had another dependency issue, different packages, but perhaps this works for you too.

I had the same issue on mac os, removing the version numbers from the file fixed it for me, pip sorted out everything.

@w0rmr1d3r Many thank, but finally went to the docker version, which works fine. We’ll try to install it on my MacOs next time i need it !

@serialp the python issue has been resolved! It should work now, have you tried it again?

Is this on macOS Big Sur? Could be this Python issue: https://bugs.python.org/issue42504

I installed CloudMapper without issues on an Ubuntu system last week.