salt: pip_state: An importable Python 2 pip module is required but could not be found on your system. This usually means that the system's pip package is not installed properly.

Description of Issue/Question

pip.installed fails to run:

----------
          ID: devpi_venv_pkgs
    Function: pip.installed
      Result: False
     Comment: An importable Python 2 pip module is required but could not be found on your system. This usually means that the system's pip package is not installed properly.
              An importable Python 2 pip module is required but could not be found on your system. This usually means that the system's pip package is not installed properly.
     Started: 16:46:18.340387
    Duration: 1.242 ms
     Changes:   

Setup

Relevant parts:

  devpi_pip_pkg:
    __env__: base
    __sls__: venvs.webapps.devpi
    pkg:
    - name: py35-pip
    - require:
      - pkg: devpi_python_pkg
    - installed
    - order: 10007

  devpi_python_pkg:
    __env__: base
    __sls__: venvs.webapps.devpi
    pkg:
    - name: python35
    - installed
    - order: 10005

  devpi_venv:
    __env__: base
    __sls__: venvs.webapps.devpi
    virtualenv:
    - name: /usr/local/venvs/devpi
    - venv_bin: /usr/local/bin/pyvenv-3.5
    - system_site_packages: false
    - user: www-devpi
    - clear: false
    - pip_upgrade: true
    - pip_pkgs:
      - pip
      - setuptools
      - certifi
    - require:
      - file: /usr/local/venvs/devpi
      - user: www-devpi
      - pkg: devpi_python_pkg
      - pkg: postgresql93-client
    - managed
    - order: 10006

  devpi_venv_pkgs:
    __env__: base
    __sls__: venvs.webapps.devpi
    pip:
    - pkgs:
      - devpi-server
      - devpi-web
    - user: www-devpi
    - cwd: /usr/local/venvs/devpi
    - bin_env: /usr/local/venvs/devpi/bin/pip
    - reload_modules: true
    - require:
      - virtualenv: devpi_venv
      - pkg: devpi_pip_pkg
    - installed
    - order: 10008

This is my complete sls:

{% if cfg.lang == 'python' %}

##########
# PYTHON #
##########

# Python packages
{{ venv }}_python_pkg:
  pkg.installed:
    - name: python{{ cfg.python_version|string|replace('.', '') }}

# Python virtualenv (only needed for Python 2)
{% if cfg.python_version == 2.7 %}
{{ venv }}_virtualenv_pkg:
  pkg.installed:
    - name: py27-virtualenv
{% endif %}

# Install the virtualenv
{{ venv }}_venv:
  virtualenv.managed:
    - name: {{ cfg.full_path }}
    - venv_bin: {{ cfg.venv_builder }}
    - system_site_packages: False
    - user: {{ cfg.user }}
    - clear: False
    - pip_upgrade: True
    - pip_pkgs:
      - pip
      - setuptools
      - certifi
    - require:
      - file: {{ cfg.full_path }}
      - user: {{ cfg.user }}
      - pkg: {{ venv }}_python_pkg
      - pkg: {{ salt['pillar.get']('postgresql:lookup:pkg_client') }}

{% if cfg.pip_pkgs is defined %}
{{ venv }}_pip_pkg:
  pkg.installed:
    - name: py{{ cfg.python_version|string|replace('.', '') }}-pip
    - require:
      - pkg: {{ venv }}_python_pkg

{{ venv }}_venv_pkgs:
  pip.installed:
    - pkgs:
        {{ cfg.pip_pkgs|yaml }}
    - user: {{ cfg.user }}
    - cwd: {{ cfg.full_path }}
    - bin_env: {{ cfg.pip }}
    - reload_modules: True
    - require:
      - virtualenv: {{ venv }}_venv
      - pkg: {{ venv }}_pip_pkg
{% endif %}

# Upgrade pip,setuptools,certifi to latest version (once the virtualenv is installed)
#{{ venv }}_upgrade_pip:
#  cmd.run:
#    - name: {{ cfg.pip }} --no-cache-dir install -U pip setuptools certifi
#    - runas: {{ cfg.user }}
#    - reload_modules: True
#    - require:
#      - virtualenv: {{ venv }}_venv

Steps to Reproduce Issue

Any pip.installed with Python 3 (haven’t tested with Python 2) should fail, even when the pip package is installed globally (which should not be needed anyway).

This is because of https://github.com/saltstack/salt/blob/develop/salt/states/pip_state.py#L112-L119

Versions Report

salt% salt --versions-report
Salt Version:
           Salt: 2017.7.2
 
Dependency Versions:
           cffi: 1.11.2
       cherrypy: Not Installed
       dateutil: 2.6.1
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: 1.5.0
         Jinja2: 2.10
        libgit2: Not Installed
        libnacl: 1.5.2
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.7
   mysql-python: Not Installed
      pycparser: 2.10
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.14 (default, Jan  8 2018, 23:05:24)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 16.0.3
           RAET: 0.6.5
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.2
            ZMQ: 4.2.2
 
System Versions:
           dist:   
         locale: UTF-8
        machine: amd64
        release: 10.3-RELEASE
         system: FreeBSD
        version: Not Installed

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 24 (19 by maintainers)

Commits related to this issue

Most upvoted comments

Still there in 3000.3(!). However, this time I even want to install Python 3 modules on a minion running Python3 Salt after explicitely installing python3-pip AND specifying bin_env: /usr/bin/pip3. Please reopen.

FWIW, this is happening because the Python2 error message is hardcoded here regardless of what Python version you’re running salt-minion in.

Also, the HAS_PIP flag is set based on whether we succeeded to import pip in the interpreter that runs salt-minion (i.e. system python3 in case of @dhs-rec and mine), not based on whether there’s pip or pip3 installed in the target bin_env virtualenv.

The workaround for me (Salt 3002.5 running on Ubuntu 20.04 with stock Python 3.8) is installing python3-pip so that the pip state is able to import pip and happily continue.

mmh… bug still exists / something else broken?

          ID: awscli-pip
    Function: pip.installed
        Name: awscli
      Result: False
     Comment: An importable Python 2 pip module is required but could not be found on your system. This usually means that the system's pip package is not installed properly.
     Started: 17:54:46.282569
    Duration: 709.242 ms
     Changes:

tested with:

# pip --version
pip 9.0.1 from /usr/lib/python2.7/dist-packages (python 2.7)

# salt-call --version
salt-call 2017.7.8 (Nitrogen)
# salt-call --versions-report
Salt Version:
           Salt: 2017.7.8

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.5.3
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.9.4
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.8
   mysql-python: 1.3.7
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.13 (default, Sep 26 2018, 18:42:22)
   python-gnupg: Not Installed
         PyYAML: 3.12
          PyZMQ: 16.0.2
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.4.3
            ZMQ: 4.2.1

System Versions:
           dist: debian 9.5
         locale: UTF-8
        machine: x86_64
        release: 4.9.0-8-amd64
         system: Linux
        version: debian 9.5

and its already setup ???

# pip install awscli
Requirement already satisfied: awscli in /usr/local/lib/python2.7/dist-packages
Requirement already satisfied: colorama<=0.3.9,>=0.2.5 in /usr/lib/python2.7/dist-packages (from awscli)
Requirement already satisfied: rsa<=3.5.0,>=3.1.2 in /usr/local/lib/python2.7/dist-packages (from awscli)
Requirement already satisfied: s3transfer<0.2.0,>=0.1.12 in /usr/local/lib/python2.7/dist-packages (from awscli)
Requirement already satisfied: PyYAML<=3.13,>=3.10 in /usr/lib/python2.7/dist-packages (from awscli)
Requirement already satisfied: docutils>=0.10 in /usr/local/lib/python2.7/dist-packages (from awscli)
Requirement already satisfied: botocore==1.12.39 in /usr/local/lib/python2.7/dist-packages (from awscli)
Requirement already satisfied: pyasn1>=0.1.3 in /usr/lib/python2.7/dist-packages (from rsa<=3.5.0,>=3.1.2->awscli)
Requirement already satisfied: futures<4.0.0,>=2.2.0; python_version == "2.6" or python_version == "2.7" in /usr/lib/python2.7/dist-packages (from s3transfer<0.2.0,>=0.1.12->awscli)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /usr/local/lib/python2.7/dist-packages (from botocore==1.12.39->awscli)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1; python_version >= "2.7" in /usr/lib/python2.7/dist-packages (from botocore==1.12.39->awscli)
Requirement already satisfied: urllib3<1.25,>=1.20; python_version == "2.7" in /usr/local/lib/python2.7/dist-packages (from botocore==1.12.39->awscli)

Both releases have been branched. Once the 2018.3.1 tests pass then there will be releases

https://jenkinsci.saltstack.com/job/2018.3.1/

Also, there is one last thing for 2017.7.6 and tornado 5.0 on python3 >=3.5

I’d like to suggest that the version incompatibilities noted here (namely, that pip2 >= 10.* is incompatible w/ 2018.3.0) be added to the docs: https://docs.saltstack.com/en/latest/ref/modules/all/salt.modules.pip.html

EDIT: I see that the docs are hosted publicly; I’ll make a PR.

thanks yeah got that 😃 appreciate the feedback on sunday mate! keep up the amazing work

Ah yes, your previous comment about that is now more clear to me. Using pip internals means salt breaks when those internals changes. Is there a way to do the version comparison using the executable instead of pip’s internals?