salt: virtualenv.managed does not use env_vars when creating the virtualenv

Description of Issue/Question

We use Salt to manage internal hosts that don’t have external access to sites like PyPI, but we so however run an internal PyPI mirror. This means when we run virtualenv to create a new virtualenv on one of these internal hosts they need to point to the internal PyPI mirror to fetch any needed python packages (ie. setuptools, pip). Usually we do this by setting the PIP_INDEX_URL environment variable before running virtualenv.

I have tried passing Salt the correct PIP_INDEX_URL environment variable using env_vars but Salt doesn’t seem to use these env_vars when invoking virtualenv.

Setup

virtualenv:
  pkg.installed:
    - name: python-virtualenv

salt-cloud-venv:
  virtualenv.managed:
    - name: /opt/salt-cloud/
    - system_site_packages: True
    - index_url: https://<internal mirror>
    - pip_pkgs:
      - pyvmomi >= 6.0.0.2016.6
    - env_vars:
        PIP_INDEX_URL: https://<internal mirror>
    - require:
      - pkg: virtualenv

Steps to Reproduce Issue

salt-call state.apply --output-diff

[ERROR   ] Command '[u'/usr/bin/virtualenv', u'--system-site-packages', u'/opt/salt-cloud/']' failed with return code: 1
[ERROR   ] stdout: New python executable in /opt/salt-cloud/bin/python2
Also creating executable in /opt/salt-cloud/bin/python
Installing setuptools, pkg_resources, pip, wheel...
  Complete output from command /opt/salt-cloud/bin/python2 - setuptools pkg_resources pip wheel:
  Collecting setuptools
Exception:
Traceback (most recent call last):
  File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/commands/install.py", line 353, in run
    wb.build(autobuilding=True)
  File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/req/req_set.py", line 554, in _prepare_file
    require_hashes
  File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/req/req_install.py", line 278, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 465, in find_requirement
    all_candidates = self.find_all_candidates(req.name)
  File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 423, in find_all_candidates
    for page in self._get_pages(url_locations, project_name):
  File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 568, in _get_pages
    page = self._get_page(location)
  File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 683, in _get_page
    return HTMLPage.get_page(link, session=self.session)
  File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 792, in get_page
    "Cache-Control": "max-age=600",
  File "/opt/salt-cloud/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 501, in get
    return self.request('GET', url, **kwargs)
  File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/download.py", line 386, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/opt/salt-cloud/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/opt/salt-cloud/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/opt/salt-cloud/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/adapter.py", line 47, in send
    resp = super(CacheControlAdapter, self).send(request, **kw)
  File "/opt/salt-cloud/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/adapters.py", line 423, in send
    timeout=timeout
  File "/opt/salt-cloud/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 643, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/opt/salt-cloud/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/util/retry.py", line 315, in increment
    total -= 1
TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'
----------------------------------------
...Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[ERROR   ] stderr: Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/virtualenv.py", line 2375, in <module>
    main()
  File "/usr/lib/python3/dist-packages/virtualenv.py", line 724, in main
    symlink=options.symlink)
  File "/usr/lib/python3/dist-packages/virtualenv.py", line 992, in create_environment
    download=download,
  File "/usr/lib/python3/dist-packages/virtualenv.py", line 922, in install_wheel
    call_subprocess(cmd, show_stdout=False, extra_env=env, stdin=SCRIPT)
  File "/usr/lib/python3/dist-packages/virtualenv.py", line 817, in call_subprocess
    % (cmd_desc, proc.returncode))
OSError: Command /opt/salt-cloud/bin/python2 - setuptools pkg_resources pip wheel failed with error code 2
[ERROR   ] retcode: 1
[ERROR   ] New python executable in /opt/salt-cloud/bin/python2
Also creating executable in /opt/salt-cloud/bin/python
Installing setuptools, pkg_resources, pip, wheel...
  Complete output from command /opt/salt-cloud/bin/python2 - setuptools pkg_resources pip wheel:
  Collecting setuptools
Exception:
Traceback (most recent call last):
  File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/commands/install.py", line 353, in run
    wb.build(autobuilding=True)
  File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/req/req_set.py", line 554, in _prepare_file
    require_hashes
  File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/req/req_install.py", line 278, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 465, in find_requirement
    all_candidates = self.find_all_candidates(req.name)
  File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 423, in find_all_candidates
    for page in self._get_pages(url_locations, project_name):
  File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 568, in _get_pages
    page = self._get_page(location)
  File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 683, in _get_page
    return HTMLPage.get_page(link, session=self.session)
  File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 792, in get_page
    "Cache-Control": "max-age=600",
  File "/opt/salt-cloud/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 501, in get
    return self.request('GET', url, **kwargs)
  File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/download.py", line 386, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/opt/salt-cloud/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/opt/salt-cloud/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/opt/salt-cloud/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/adapter.py", line 47, in send
    resp = super(CacheControlAdapter, self).send(request, **kw)
  File "/opt/salt-cloud/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/adapters.py", line 423, in send
    timeout=timeout
  File "/opt/salt-cloud/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 643, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/opt/salt-cloud/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/util/retry.py", line 315, in increment
    total -= 1
TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'
----------------------------------------
...Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/virtualenv.py", line 2375, in <module>
    main()
  File "/usr/lib/python3/dist-packages/virtualenv.py", line 724, in main
    symlink=options.symlink)
  File "/usr/lib/python3/dist-packages/virtualenv.py", line 992, in create_environment
    download=download,
  File "/usr/lib/python3/dist-packages/virtualenv.py", line 922, in install_wheel
    call_subprocess(cmd, show_stdout=False, extra_env=env, stdin=SCRIPT)
  File "/usr/lib/python3/dist-packages/virtualenv.py", line 817, in call_subprocess
    % (cmd_desc, proc.returncode))
OSError: Command /opt/salt-cloud/bin/python2 - setuptools pkg_resources pip wheel failed with error code 2
local:
----------
          ID: salt-cloud-venv
    Function: virtualenv.managed
        Name: /opt/salt-cloud/
      Result: False
     Comment: New python executable in /opt/salt-cloud/bin/python2
              Also creating executable in /opt/salt-cloud/bin/python                                                                
              Installing setuptools, pkg_resources, pip, wheel...                                                                   
                Complete output from command /opt/salt-cloud/bin/python2 - setuptools pkg_resources pip wheel:                      
                Collecting setuptools                                                                                               
              Exception:                                                                                                            
              Traceback (most recent call last):                                                                                    
                File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/basecommand.py", line 215, in main                
                  status = self.run(options, args)                                                                                  
                File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/commands/install.py", line 353, in run            
                  wb.build(autobuilding=True)                                                                                       
                File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/wheel.py", line 749, in build                     
                  self.requirement_set.prepare_files(self.finder)                                                                   
                File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/req/req_set.py", line 380, in prepare_files       
                  ignore_dependencies=self.ignore_dependencies))                                                                    
                File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/req/req_set.py", line 554, in _prepare_file       
                  require_hashes                                                                                                    
                File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/req/req_install.py", line 278, in populate_link   
                  self.link = finder.find_requirement(self, upgrade)                                                                
                File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 465, in find_requirement          
                  all_candidates = self.find_all_candidates(req.name)                                                               
                File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 423, in find_all_candidates       
                  for page in self._get_pages(url_locations, project_name):                                                         
                File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 568, in _get_pages                
                  page = self._get_page(location)                                                                                   
                File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 683, in _get_page                 
                  return HTMLPage.get_page(link, session=self.session)                                                              
                File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/index.py", line 792, in get_page                  
                  "Cache-Control": "max-age=600",                                                                                   
                File "/opt/salt-cloud/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 501, in get                                                                                                                                  
                  return self.request('GET', url, **kwargs)                                                                         
                File "/usr/share/python-wheels/pip-9.0.1-py2.py3-none-any.whl/pip/download.py", line 386, in request                
                  return super(PipSession, self).request(method, url, *args, **kwargs)                                              
                File "/opt/salt-cloud/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 488, in request                                                                                                                              
                  resp = self.send(prep, **send_kwargs)                                                                             
                File "/opt/salt-cloud/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 609, in send                                                                                                                                 
                  r = adapter.send(request, **kwargs)                                                                               
                File "/opt/salt-cloud/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/adapter.py", line 47, in send                                                                                                                           
                  resp = super(CacheControlAdapter, self).send(request, **kw)                                                       
                File "/opt/salt-cloud/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/adapters.py", line 423, in send                                                                                                                                 
                  timeout=timeout                                                                                                   
                File "/opt/salt-cloud/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 643, in urlopen                                                                                                                          
                  _stacktrace=sys.exc_info()[2])                                                                                    
                File "/opt/salt-cloud/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/util/retry.py", line 315, in increment                                                                                                                            
                  total -= 1                                                                                                        
              TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'                                                      
              ----------------------------------------                                                                              
              ...Installing setuptools, pkg_resources, pip, wheel...done.                                                           
              Running virtualenv with interpreter /usr/bin/python2Traceback (most recent call last):                                
                File "/usr/lib/python3/dist-packages/virtualenv.py", line 2375, in <module>
                  main()
                File "/usr/lib/python3/dist-packages/virtualenv.py", line 724, in main
                  symlink=options.symlink)
                File "/usr/lib/python3/dist-packages/virtualenv.py", line 992, in create_environment
                  download=download,
                File "/usr/lib/python3/dist-packages/virtualenv.py", line 922, in install_wheel
                  call_subprocess(cmd, show_stdout=False, extra_env=env, stdin=SCRIPT)
                File "/usr/lib/python3/dist-packages/virtualenv.py", line 817, in call_subprocess
                  % (cmd_desc, proc.returncode))
              OSError: Command /opt/salt-cloud/bin/python2 - setuptools pkg_resources pip wheel failed with error code 2
     Started: 12:41:12.314248
    Duration: 2210.273 ms
     Changes:   

Summary for local
-------------
Succeeded: 1
Failed:     1
-------------
Total states run:     2
Total run time:    1.920 s

The error would appear to be due to the fact that the internal host can’t connect to the PyPI to download the setuptools package. If virtualenv where to have the env_vars variables set before it was invoked then the issue would not occur.

Versions Report

Salt Version:
           Salt: 2018.3.2
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: 3.5.0
       dateutil: 2.5.3
      docker-py: Not Installed
          gitdb: 2.0.0
      gitpython: 2.1.1
          ioflo: Not Installed
         Jinja2: 2.9.4
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.8
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.13 (default, Nov 24 2017, 17:33:09)
   python-gnupg: 0.3.9
         PyYAML: 3.12
          PyZMQ: 16.0.2
           RAET: Not Installed
          smmap: 2.0.1
        timelib: Not Installed
        Tornado: 4.4.3
            ZMQ: 4.2.1
 
System Versions:
           dist: debian 9.5 
         locale: UTF-8
        machine: x86_64
        release: 4.9.0-6-amd64
         system: Linux
        version: debian 9.5

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 2
  • Comments: 15 (7 by maintainers)

Most upvoted comments

Is there any way to stop issues being marked as stale every single month… it seems a little bit aggressive.

+1
jerrykan on Apr 8, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

+1
stale[bot] on Jan 9, 2020

@garethgreenaway Thanks for looking at this. The part where Salt calls pip directly seems to work fine. The issue is when Salt calls virtualenv to create the new virtualenv. As best as I can tell virtualenv invokes the system pip to install setuptools and pip within the virtualenv when it is first created. However because the environment variables aren’t set before Salt invokes virtualenv, when virtualenv invokes the system pip it doesn’t know to use the internal PyPI mirror.

If I manually create the virtualenv first using something like cmd.run and then use virtualenv.managed to mange the virtualenv, then everything seems to work OK. ie:

virtualenv:
  pkg.installed:
    - name: python-virtualenv

salt-cloud-venv:
  cmd.run:
    - name: virtualenv --system-site-packages /opt/salt-cloud
    - unless: test -d /opt/salt-cloud
    - env:
      - PIP_INDEX_URL: https://<internal mirror>
    - require:
      - pkg: virtualenv

  virtualenv.managed:
    - name: /opt/salt-cloud/
    - system_site_packages: True
    - index_url: https://<internal mirror>
    - pip_pkgs:
      - pyvmomi >= 6.0.0.2016.6
    - env_vars:
        PIP_INDEX_URL: https://<internal mirror>
    - require:
      - cmd: salt-cloud-venv
+1
jerrykan on Sep 14, 2018
Read more comments on GitHub
← salt: version param in pkg.installed broken in 2016.11.8/2017.7.2 in EL6-7
salt: win_dacl module stacktrace: NameError: name 'ntsecuritycon' is not defined →