vsphere-automation-sdk-python: error "Supplied requirement is not installable" installing vsphere-automation-sdk-python v7.0.1 from git/pipenv

TL;DR; does the sdk install require the community.vmware ansible collection requirements.txt to be installed first?

Moving issue from community.vmware ansible collection, I am getting errors installing vsphere-automation-sdk-python from its most recent git tag v7.0.1, using ansible 2.10.7, pyvmomi 7.0.1, python 3.7.3, and pipenv 2020.11.15, and the following Pipfile:

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
# See Pipfile.lock for all package details
ansible = "~=2.10"
# vsphere sdk docs say that it installs pyvmomi, so exclude it here:
# https://docs.ansible.com/ansible/latest/scenario_guides/vmware_scenarios/vmware_intro.html#requirements
# pyvmomi = "~=7.0.1"
setuptools = "~=53.0.0"
# must be on one line
vsphere-automation-sdk-python = { git = "https://github.com/vmware/vsphere-automation-sdk-python.git", editable = true, ref = "v7.0.1.0" }

[requires]
python_version = "3"

As discussed here and here, it appears that my syntax is correct for pipenv installing a package from git repo.

The error is as follows:

⠧ Locking...✘ Locking Failed! 
10:04:41  Traceback (most recent call last):
10:04:41    File "/usr/local/lib/python3.7/dist-packages/pipenv/resolver.py", line 764, in <module>
10:04:41      main()
10:04:41    File "/usr/local/lib/python3.7/dist-packages/pipenv/resolver.py", line 760, in main
10:04:41      dev=parsed.dev)
10:04:41    File "/usr/local/lib/python3.7/dist-packages/pipenv/resolver.py", line 741, in _main
10:04:41      resolve_packages(pre, clear, verbose, system, write, requirements_dir, packages, dev)
10:04:41    File "/usr/local/lib/python3.7/dist-packages/pipenv/resolver.py", line 709, in resolve_packages
10:04:41      requirements_dir=requirements_dir,
10:04:41    File "/usr/local/lib/python3.7/dist-packages/pipenv/resolver.py", line 692, in resolve
10:04:41      req_dir=requirements_dir
10:04:41    File "/usr/local/lib/python3.7/dist-packages/pipenv/utils.py", line 1403, in resolve_deps
10:04:41      req_dir=req_dir,
10:04:41    File "/usr/local/lib/python3.7/dist-packages/pipenv/utils.py", line 1106, in actually_resolve_deps
10:04:41      deps, index_lookup, markers_lookup, project, sources, req_dir, clear, pre
10:04:41    File "/usr/local/lib/python3.7/dist-packages/pipenv/utils.py", line 664, in create
10:04:41      pre=pre, clear=clear
10:04:41    File "/usr/local/lib/python3.7/dist-packages/pipenv/utils.py", line 463, in get_metadata
10:04:41      req, resolver=transient_resolver
10:04:41    File "/usr/local/lib/python3.7/dist-packages/pipenv/utils.py", line 571, in get_deps_from_req
10:04:41      new_req, _, _ = cls.parse_line(line)
10:04:41    File "/usr/local/lib/python3.7/dist-packages/pipenv/utils.py", line 494, in parse_line
10:04:41      req = Requirement.from_line(line)
10:04:41    File "/usr/local/lib/python3.7/dist-packages/pipenv/vendor/requirementslib/models/requirements.py", line 2676, in from_line
10:04:41      parsed_line = Line(line)
10:04:41    File "/usr/local/lib/python3.7/dist-packages/pipenv/vendor/requirementslib/models/requirements.py", line 171, in __init__
10:04:41      self.parse()
10:04:41    File "/usr/local/lib/python3.7/dist-packages/pipenv/vendor/requirementslib/models/requirements.py", line 1301, in parse
10:04:41      "Supplied requirement is not installable: {0!r}".format(self.line)
10:04:41  pipenv.vendor.requirementslib.exceptions.RequirementError: Supplied requirement is not installable: 'vmc-client-bindings@ file://localhost//tmp/pipenv-mygb68pv-src/vsphere-automation-sdk-python/lib/vmc-client-bindings/vmc_client_bindings-1.29.0-py2.py3-none-any.whl'

Note that the wheel shown in the error message above is noto always vmc-client-bindings; it changes each time I do a fresh pipenv install. I’ve seen it show nsx-vmc-policy-python-sdk and vapi-client-bindings as well…

Pipenv is creating the virtual environment from the Pipfile from scratch in a Jenkins job which runs a shell script which runs the following commands. Note that it is not getting past the first command as the pipenv install is failing.

                            pipenv install  # installs packages from Pipfile
                            pipenv run ansible-galaxy install -fr roles/requirements.yml
                            pipenv run ansible-galaxy collection install -r roles/requirements.yml
                            echo "installing special requirements for community.vmware collection, per https://github.com/ansible-collections/community.vmware#installing-required-libraries-and-sdk..."
                            # Note: collection_paths is defined in local ansible.cfg to be project root
                            pipenv install -r ./ansible_collections/community/vmware/requirements.txt
                            pipenv graph

I want to run an ansible-playbook that uses community.vmware.vmware_guest and here I am simply following the collection’s instructions for installing its pre-requisites. (BTW, I read here that the sdk includes pyvmomi, so IMO your docs shouldn’t separately include pyvmomi as a dependency.)

This feels like a sequencing issue, and the full breadth of this collection’s dependencies are not documented very well for someone like myself trying to automate the process. Your docs here say that in addition to the sdk package, you also have to pip install the requirements.txt that is included in the installed collection. As you can see in my script above, I’m doing this. But only attempting to do this after having installed the venv through Pipfile. Is this order of operations an issue?

I.e. does the sdk install require the collection requirements to be installed first?

COMPONENT NAME

vsphere-automation-sdk-python

ANSIBLE VERSION
09:40:06  ansible 2.10.6
09:40:06    config file = /home/jenkins/.jenkins/workspace/nkins-testing-vsphere-machines_4/ansible/projects/vsphere-engineering-services-template/ansible.cfg
09:40:06    configured module search path = ['/home/jenkins/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
09:40:06   ansible python module location = /home/jenkins/.local/share/virtualenvs/vsphere-engineering-services-template-Y97Wbu3u/lib/python3.7/site-packages/ansible
09:40:06    executable location = /home/jenkins/.local/share/virtualenvs/vsphere-engineering-services-template-Y97Wbu3u/bin/ansible
09:40:06    python version = 3.7.3 (default, Jul 25 2020, 13:03:44) [GCC 8.3.0]
CONFIGURATION
10:22:51  + pipenv run ansible-config dump --only-changed
10:22:52  ANSIBLE_PIPELINING(/home/jenkins/.jenkins/workspace/nkins-testing-vsphere-machines_4/ansible/projects/vsphere-engineering-services-template/ansible.cfg) = True
10:22:52  ANSIBLE_SSH_ARGS(/home/jenkins/.jenkins/workspace/nkins-testing-vsphere-machines_4/ansible/projects/vsphere-engineering-services-template/ansible.cfg) = -o ControlMaster=auto -o ControlPersist=60s -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no
10:22:52  COLLECTIONS_PATHS(/home/jenkins/.jenkins/workspace/nkins-testing-vsphere-machines_4/ansible/projects/vsphere-engineering-services-template/ansible.cfg) = ['/home/jenkins/.jenkins/workspace/nkins-testing-vsphere-machines_4/ansible/projects/vsphere-engineering-services-template']
10:22:52  DEFAULT_CALLBACK_WHITELIST(/home/jenkins/.jenkins/workspace/nkins-testing-vsphere-machines_4/ansible/projects/vsphere-engineering-services-template/ansible.cfg) = ['timer']
10:22:52  DEFAULT_FORKS(/home/jenkins/.jenkins/workspace/nkins-testing-vsphere-machines_4/ansible/projects/vsphere-engineering-services-template/ansible.cfg) = 20
10:22:52  DEFAULT_HOST_LIST(/home/jenkins/.jenkins/workspace/nkins-testing-vsphere-machines_4/ansible/projects/vsphere-engineering-services-template/ansible.cfg) = ['/home/jenkins/.jenkins/workspace/nkins-testing-vsphere-machines_4/ansible/projects/vsphere-engineering-services-template/inventories/testing/hosts']
10:22:52  DEFAULT_KEEP_REMOTE_FILES(/home/jenkins/.jenkins/workspace/nkins-testing-vsphere-machines_4/ansible/projects/vsphere-engineering-services-template/ansible.cfg) = True
10:22:52  DEFAULT_LOAD_CALLBACK_PLUGINS(/home/jenkins/.jenkins/workspace/nkins-testing-vsphere-machines_4/ansible/projects/vsphere-engineering-services-template/ansible.cfg) = True
10:22:52  DEFAULT_LOG_PATH(/home/jenkins/.jenkins/workspace/nkins-testing-vsphere-machines_4/ansible/projects/vsphere-engineering-services-template/ansible.cfg) = /home/jenkins/.jenkins/workspace/nkins-testing-vsphere-machines_4/ansible/projects/vsphere-engineering-services-template/ansible.log
10:22:52  DEFAULT_MANAGED_STR(/home/jenkins/.jenkins/workspace/nkins-testing-vsphere-machines_4/ansible/projects/vsphere-engineering-services-template/ansible.cfg) = #Ansible managed by user {uid} on {host} from template file:
10:22:52--
10:22:52  DEFAULT_POLL_INTERVAL(/home/jenkins/.jenkins/workspace/nkins-testing-vsphere-machines_4/ansible/projects/vsphere-engineering-services-template/ansible.cfg) = 5
10:22:52  DEFAULT_REMOTE_USER(/home/jenkins/.jenkins/workspace/nkins-testing-vsphere-machines_4/ansible/projects/vsphere-engineering-services-template/ansible.cfg) = ansible
10:22:52  DEFAULT_ROLES_PATH(/home/jenkins/.jenkins/workspace/nkins-testing-vsphere-machines_4/ansible/projects/vsphere-engineering-services-template/ansible.cfg) = ['/home/jenkins/.jenkins/workspace/nkins-testing-vsphere-machines_4/ansible/projects/vsphere-engineering-services-template/roles', '/home/jenkins/.jenkins/workspace/nkins-testing-vsphere-machines_4/ansible/roles']
10:22:52  DEFAULT_STDOUT_CALLBACK(/home/jenkins/.jenkins/workspace/nkins-testing-vsphere-machines_4/ansible/projects/vsphere-engineering-services-template/ansible.cfg) = yaml
10:22:52  HOST_KEY_CHECKING(/home/jenkins/.jenkins/workspace/nkins-testing-vsphere-machines_4/ansible/projects/vsphere-engineering-services-template/ansible.cfg) = False
10:22:52  INTERPRETER_PYTHON(/home/jenkins/.jenkins/workspace/nkins-testing-vsphere-machines_4/ansible/projects/vsphere-engineering-services-template/ansible.cfg) = /usr/bin/python3
10:22:52  RETRY_FILES_ENABLED(/home/jenkins/.jenkins/workspace/nkins-testing-vsphere-machines_4/ansible/projects/vsphere-engineering-services-template/ansible.cfg) = False
OS / ENVIRONMENT

Debian 10.7

STEPS TO REPRODUCE

Can’t even run playbook that uses vmware_guest bc dependencies of this collection cannot be installed.

EXPECTED RESULTS

I expect sdk to get installed by pipenv and then i can install this collection.

ACTUAL RESULTS

see above

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 1
  • Comments: 18 (5 by maintainers)

Most upvoted comments

Assuming it’s tracked elsewhere that this project should create an installable package on a public pypi, this issue can be resolved, since as I mentioned here I’m able to at least install it, but the possible approaches I can use to do so are very limited: It appears to only work via Direct installation via pip, referencing a release archive file or a branch name. It does not work when trying to reference this package by name or any which way when using pipenv. (I switched to using pyenv/virtualenv just for this project.)

+1
timblaktu on Apr 14, 2021

Now using python 3.9.0, I’m getting success but only when I pipenv install a release archive file, using this package type/syntax, vsphere-automation-sdk = {file = "https://github.com/vmware/vsphere-automation-sdk-python/archive/v7.0.1.0.tar.gz"}.

Here is the full Pipfile I’m using in this successful case:

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
ansible = "~=2.10"
setuptools = "~=53.0.0"
# NB: we install the sdk here, instead of using the requirements.txt file provided by the community.vmware collection,-
#     which isn't in the correct syntax for pipenv
# DOESN'T WORK:
# vsphere-automation-sdk-python = { git = "https://github.com/vmware/vsphere-automation-sdk-python.git", editable = true, ref = "v7.0.1.0" }
vsphere-automation-sdk = {file = "https://github.com/vmware/vsphere-automation-sdk-python/archive/v7.0.1.0.tar.gz"}

[requires]
python_version = "3"

Even though this succeeds, strangely, a subsequent graph of what got installed shows an sdk version of 1.39.0. Is this discrepancy between the git repo tag/version and the reported python package version expected?

07:41:53  + pipenv graph
07:41:55  ansible==2.10.7
07:41:55    - ansible-base [required: >=2.10.5,<2.11, installed: 2.10.6]
07:41:55      - cryptography [required: Any, installed: 3.4.6]
07:41:55        - cffi [required: >=1.12, installed: 1.14.5]
07:41:55          - pycparser [required: Any, installed: 2.20]
07:41:55      - jinja2 [required: Any, installed: 2.11.3]
07:41:55        - MarkupSafe [required: >=0.23, installed: 1.1.1]
07:41:55      - packaging [required: Any, installed: 20.9]
07:41:55        - pyparsing [required: >=2.0.2, installed: 2.4.7]
07:41:55      - PyYAML [required: Any, installed: 5.4.1]
07:41:55  vSphere-Automation-SDK==1.39.0
07:41:55    - lxml [required: >=4.3.0, installed: 4.6.2]
07:41:55    - nsx-policy-python-sdk [required: Any, installed: ?]
07:41:55    - nsx-python-sdk [required: Any, installed: ?]
07:41:55    - nsx-vmc-aws-integration-python-sdk [required: Any, installed: ?]
07:41:55    - nsx-vmc-policy-python-sdk [required: Any, installed: ?]
07:41:55    - pyVmomi [required: >=6.7, installed: 7.0.1]
07:41:55      - requests [required: >=2.3.0, installed: 2.25.1]
07:41:55        - certifi [required: >=2017.4.17, installed: 2020.12.5]
07:41:55        - chardet [required: >=3.0.2,<5, installed: 4.0.0]
07:41:55        - idna [required: >=2.5,<3, installed: 2.10]
07:41:55        - urllib3 [required: >=1.21.1,<1.27, installed: 1.26.3]
07:41:55      - six [required: >=1.7.3, installed: 1.15.0]
07:41:55    - suds-jurko [required: Any, installed: 0.6]
07:41:55    - vapi-client-bindings [required: Any, installed: ?]
07:41:55    - vapi-common-client [required: Any, installed: ?]
07:41:55    - vapi-runtime [required: Any, installed: ?]
07:41:55    - vmc-client-bindings [required: Any, installed: ?]
07:41:55    - vmc-draas-client-bindings [required: Any, installed: ?]

Everything else fails. For example, Pipenv installing with this package syntax:

vsphere-automation-sdk-python = { git = "https://github.com/vmware/vsphere-automation-sdk-python.git", editable = true, ref = "v7.0.1.0" }

produces the same error I was getting earlier:

Building requirements...
07:35:27  
Resolving dependencies...
07:35:59  
FAIL
07:35:59  Traceback (most recent call last):
07:35:59    File "/home/jenkins/.local/lib/python3.9/site-packages/pipenv/resolver.py", line 764, in <module>
07:35:59      main()
07:35:59    File "/home/jenkins/.local/lib/python3.9/site-packages/pipenv/resolver.py", line 758, in main
07:35:59      _main(parsed.pre, parsed.clear, parsed.verbose, parsed.system, parsed.write,
07:35:59    File "/home/jenkins/.local/lib/python3.9/site-packages/pipenv/resolver.py", line 741, in _main
07:35:59      resolve_packages(pre, clear, verbose, system, write, requirements_dir, packages, dev)
07:35:59    File "/home/jenkins/.local/lib/python3.9/site-packages/pipenv/resolver.py", line 702, in resolve_packages
07:35:59      results, resolver = resolve(
07:35:59    File "/home/jenkins/.local/lib/python3.9/site-packages/pipenv/resolver.py", line 684, in resolve
07:35:59      return resolve_deps(
07:35:59    File "/home/jenkins/.local/lib/python3.9/site-packages/pipenv/utils.py", line 1395, in resolve_deps
07:35:59      results, hashes, markers_lookup, resolver, skipped = actually_resolve_deps(
07:35:59    File "/home/jenkins/.local/lib/python3.9/site-packages/pipenv/utils.py", line 1105, in actually_resolve_deps
07:35:59      resolver = Resolver.create(
07:35:59    File "/home/jenkins/.local/lib/python3.9/site-packages/pipenv/utils.py", line 662, in create
07:35:59      constraints, skipped, index_lookup, markers_lookup = cls.get_metadata(
07:35:59    File "/home/jenkins/.local/lib/python3.9/site-packages/pipenv/utils.py", line 462, in get_metadata
07:35:59      constraint_update, lockfile_update = cls.get_deps_from_req(
07:35:59    File "/home/jenkins/.local/lib/python3.9/site-packages/pipenv/utils.py", line 571, in get_deps_from_req
07:35:59      new_req, _, _ = cls.parse_line(line)
07:35:59    File "/home/jenkins/.local/lib/python3.9/site-packages/pipenv/utils.py", line 494, in parse_line
07:35:59      req = Requirement.from_line(line)
07:35:59    File "/home/jenkins/.local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 2676, in from_line
07:35:59      parsed_line = Line(line)
07:35:59    File "/home/jenkins/.local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 171, in __init__
07:35:59      self.parse()
07:35:59    File "/home/jenkins/.local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 1300, in parse
07:35:59      raise RequirementError(
07:35:59  pipenv.vendor.requirementslib.exceptions.RequirementError: Supplied requirement is not installable: 'vmc-client-bindings@ file://localhost//tmp/pipenv-ct007amj-src/vsphere-automation-sdk-python/lib/vmc-client-bindings/vmc_client_bindings-1.29.0-py2.py3-none-any.whl'
+1
timblaktu on Mar 10, 2021
Read more comments on GitHub
← terraform-provider-vcd: VMware - Vcloud Air/ Vcloud Director: error: "* Something went wrong: error finding LoginUrl: error parsing error body for non-200 request: EOF"
vsphere-automation-sdk-python: Error when trying to list all the vm's running. →