salt: Neon: name 'pip' is not defined

Description of Issue

I’ve been testing some stuff on Neon and got the following traceback:

2019-06-24 03:25:16,909 [salt.state       :1880][INFO    ][9032] Running state [/tmp/file.txt] at time 03:25:16.909899
2019-06-24 03:25:16,910 [salt.state       :1913][INFO    ][9032] Executing state file.managed for [/tmp/file.txt]
2019-06-24 03:25:17,341 [salt.loaded.int.module.cmdmod:399 ][INFO    ][9032] Executing command 'pkg_info -Q LOCALBASE pkgin' in directory '/root'
2019-06-24 03:25:18,018 [salt.loaded.int.module.cmdmod:399 ][INFO    ][9032] Executing command [u'git', u'--version'] in directory '/root'
2019-06-24 03:25:18,225 [salt.loader      :1701][ERROR   ][9032] Failed to import states pip_state, this is due most likely to a syntax error:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 1684, in _load_module
    mod = imp.load_module(mod_namespace, fn_, fpath, desc)
  File "/usr/lib/python2.7/dist-packages/salt/states/pip_state.py", line 58, in <module>
    del pip
NameError: name 'pip' is not defined

Immediately after the traceback, I get huge amount of other exceptions:

Collapsed...

2019-06-24 03:44:50,031 [salt.loader      :1701][ERROR   ][8530] Failed to import states pkg, this is due most likely to a syntax error:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 1684, in _load_module
    mod = imp.load_module(mod_namespace, fn_, fpath, desc)
  File "/usr/lib/python2.7/dist-packages/salt/states/pkg.py", line 79, in <module>
    import logging
  File "/usr/lib/python2.7/logging/__init__.py", line 26, in <module>
    import sys, os, time, cStringIO, traceback, warnings, weakref, collections
  File "/usr/lib/python2.7/os.py", line 119, in <module>
    sys.modules['os.path'] = path
AttributeError: 'module' object has no attribute 'modules'
2019-06-24 03:44:50,034 [salt.loader      :1701][ERROR   ][8530] Failed to import states pkgbuild, this is due most likely to a syntax error:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 1684, in _load_module
    mod = imp.load_module(mod_namespace, fn_, fpath, desc)
  File "/usr/lib/python2.7/dist-packages/salt/states/pkgbuild.py", line 48, in <module>
    import logging
  File "/usr/lib/python2.7/logging/__init__.py", line 26, in <module>
    import sys, os, time, cStringIO, traceback, warnings, weakref, collections
  File "/usr/lib/python2.7/os.py", line 119, in <module>
    sys.modules['os.path'] = path
AttributeError: 'module' object has no attribute 'modules'
2019-06-24 03:44:50,046 [salt.loader      :1701][ERROR   ][8530] Failed to import states pkgrepo, this is due most likely to a syntax error:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 1684, in _load_module
    mod = imp.load_module(mod_namespace, fn_, fpath, desc)
  File "/usr/lib/python2.7/dist-packages/salt/states/pkgrepo.py", line 93, in <module>
    from salt.exceptions import CommandExecutionError, SaltInvocationError
  File "/usr/lib/python2.7/dist-packages/salt/__init__.py", line 100, in <module>
    __define_global_system_encoding_variable__()
  File "/usr/lib/python2.7/dist-packages/salt/__init__.py", line 58, in __define_global_system_encoding_variable__
    encoding = locale.getdefaultlocale()[-1]
  File "/usr/lib/python2.7/locale.py", line 535, in getdefaultlocale
    import os
  File "/usr/lib/python2.7/os.py", line 119, in <module>
    sys.modules['os.path'] = path
AttributeError: 'module' object has no attribute 'modules'

...

Setup

Ubuntu 18.04 in Vagrant (with apt-get dist-upgrade applied)
Salt 967918fa8dee8a3ae09e33133249fd9d49f8d4ca installed via bootstrap-salt.sh -P -f -g https://github.com/saltstack/salt.git -F -c /tmp git neon

Steps to Reproduce Issue

Apply the following (simplified) state using salt XXX state.apply teststate

# teststate.sls
/tmp/file.txt:
  file.managed:
    - contents: |
        Content
    - unless:
      - 'grep "Content" /tmp/file.txt'

The same state works just fine on Salt 2019.2.0.

Versions Report

sudo salt-minion --versions-report

Salt Version:
           Salt: 2019.2.0-n/a-967918f

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: Not Installed
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
         Jinja2: 2.10
        libgit2: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 3.6.8 (default, Jan 14 2019, 11:02:34)
   python-gnupg: Not Installed
         PyYAML: 3.12
          PyZMQ: 16.0.2
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.2.5

System Versions:
           dist: Ubuntu 18.04 bionic
         locale: UTF-8
        machine: x86_64
        release: 4.15.0-29-generic
         system: Linux
        version: Ubuntu 18.04 bionic

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 19 (14 by maintainers)

Most upvoted comments

With the release of the packaged 2019.2.1, I am bitten by this bug too. Is there a reason why this shouldn’t be fixed as described by https://github.com/saltstack/salt/issues/53570#issuecomment-508709838

swinkels on Sep 26, 2019

@micdobro The bug is fixed in 2019.2.2

max-arnold on Nov 29, 2019

I agree that it could be an issue with salt-bootstrap. But why it is not reproducible with 2019.2 branch? Is there something in salt-bootstrap that checks for specific salt version/branch?

max-arnold on Jul 2, 2019

@Akm0d

I think I was able to reproduce it on both versions of Python, that is why I posted mixed versions. Just did another test:

sudo salt minion1 state.apply teststate:

Minion log: ModuleNotFoundError: No module named 'pip'

2019-07-02 16:49:52,194 [salt.loader      :1701][ERROR   ][8563] Failed to import states pip_state, this is due most likely to a syntax error:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/states/pip_state.py", line 43, in <module>
    import pip
ModuleNotFoundError: No module named 'pip'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/loader.py", line 1677, in _load_module
    mod = spec.loader.load_module()
  File "<frozen importlib._bootstrap_external>", line 399, in _check_name_wrapper
  File "<frozen importlib._bootstrap_external>", line 823, in load_module
  File "<frozen importlib._bootstrap_external>", line 682, in load_module
  File "<frozen importlib._bootstrap>", line 265, in _load_module_shim
  File "<frozen importlib._bootstrap>", line 684, in _load
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/lib/python3/dist-packages/salt/states/pip_state.py", line 58, in <module>
    del pip
NameError: name 'pip' is not defined
2019-07-02 16:49:52,200 [salt.loader      :1701][ERROR   ][8563] Failed to import states pkg, this is due most likely to a syntax error:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/loader.py", line 1677, in _load_module
    mod = spec.loader.load_module()
  File "<frozen importlib._bootstrap_external>", line 399, in _check_name_wrapper
  File "<frozen importlib._bootstrap_external>", line 823, in load_module
  File "<frozen importlib._bootstrap_external>", line 682, in load_module
  File "<frozen importlib._bootstrap>", line 265, in _load_module_shim
  File "<frozen importlib._bootstrap>", line 684, in _load
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/lib/python3/dist-packages/salt/states/pkg.py", line 78, in <module>
    import fnmatch
  File "/usr/lib/python3.6/fnmatch.py", line 12, in <module>
    import os
  File "/usr/lib/python3.6/os.py", line 91, in <module>
    sys.modules['os.path'] = path
AttributeError: module 'sys' has no attribute 'modules'
2019-07-02 16:49:52,204 [salt.loader      :1701][ERROR   ][8563] Failed to import states pkgbuild, this is due most likely to a syntax error:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/loader.py", line 1677, in _load_module
    mod = spec.loader.load_module()
  File "<frozen importlib._bootstrap_external>", line 399, in _check_name_wrapper
  File "<frozen importlib._bootstrap_external>", line 823, in load_module
  File "<frozen importlib._bootstrap_external>", line 682, in load_module
  File "<frozen importlib._bootstrap>", line 265, in _load_module_shim
  File "<frozen importlib._bootstrap>", line 684, in _load
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/lib/python3/dist-packages/salt/states/pkgbuild.py", line 48, in <module>
    import logging
  File "/usr/lib/python3.6/logging/__init__.py", line 26, in <module>
    import sys, os, time, io, traceback, warnings, weakref, collections
  File "/usr/lib/python3.6/os.py", line 91, in <module>
    sys.modules['os.path'] = path
AttributeError: module 'sys' has no attribute 'modules'

salt --versions-report

Salt Version:
           Salt: 2019.2.0-n/a-a3ad595

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: Not Installed
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
         Jinja2: 2.10
        libgit2: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 3.6.8 (default, Jan 14 2019, 11:02:34)
   python-gnupg: Not Installed
         PyYAML: 3.12
          PyZMQ: 16.0.2
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.2.5

System Versions:
           dist: Ubuntu 18.04 bionic
         locale: UTF-8
        machine: x86_64
        release: 4.15.0-29-generic
         system: Linux
        version: Ubuntu 18.04 bionic

salt minion1 test.versions_report

minion1:
    Salt Version:
               Salt: 2019.2.0-n/a-a3ad595

    Dependency Versions:
               cffi: Not Installed
           cherrypy: Not Installed
           dateutil: Not Installed
          docker-py: Not Installed
              gitdb: Not Installed
          gitpython: Not Installed
             Jinja2: 2.10
            libgit2: Not Installed
           M2Crypto: Not Installed
               Mako: Not Installed
       msgpack-pure: Not Installed
     msgpack-python: 0.5.6
       mysql-python: Not Installed
          pycparser: Not Installed
           pycrypto: 2.6.1
       pycryptodome: Not Installed
             pygit2: Not Installed
             Python: 3.6.8 (default, Jan 14 2019, 11:02:34)
       python-gnupg: Not Installed
             PyYAML: 3.12
              PyZMQ: 16.0.2
              smmap: Not Installed
            timelib: Not Installed
            Tornado: 4.5.3
                ZMQ: 4.2.5

    System Versions:
               dist: Ubuntu 18.04 bionic
             locale: UTF-8
            machine: x86_64
            release: 4.15.0-29-generic
             system: Linux
            version: Ubuntu 18.04 bionic

pip executable does not exist on both master and minion (probably wasn’t installed by salt-bootstrap.sh):

bootstrap-salt.sh -P -x python3 -f -g https://github.com/saltstack/salt.git -F -c /tmp -k /tmp/minion-seed-keys -M -N git neon

max-arnold on Jul 2, 2019