salt: salt-master on CentOS 7 using the Python 3 packages cannot use pygit2 via normal means

Description of Issue/Question

As has been dealt with in a few different issues, EPEL recently changed from Python 3.4 to Python 3.6. Salt packages have been updated, and new installs of Salt using PY3 instructions here work.

However, pygit2 is not provided by EPEL or salt-py3-repo for Python 3.6. Additionally, the Python 3.6 version of Salt installed by salt-py3-repo on CentOS 7 specifies an interpreter of #!/usr/bin/python3.6 -s. The -s argument avoids loading the user’s site directory to syspath and as such, using pip to install pygit2 for Python 3.6 will not work.

It seems that salt-py3-repo should provide a python36-pygit2 package as it does some other python36-* packages, or the -s argument should be removed so that users can install required packages not provided by the repositories.

Setup

Bring up a clean install of CentOS 7
Install EPEL following the instructions here
Install the latest salt-master release (2019.2.0 as of this report) using the instructions for “Redhat / CentOS 7 PY3” here
Install pip for Python 3.6 (yum install python36-pip)
Install pygit2 via pip for Python 3.6 (pip3.6 install pygit2)

Steps to Reproduce Issue

Run salt-master -V and observe that pygit2 is unavailable
Observe that python36-pygit2 is not available from EPEL or salt-py3-repo
head -1 "$(which salt-master)" shows the restrictive interpreter that does not allow the version of pygit2 installed by pip to function for this particular version of Salt

Versions Report

Salt Version:
           Salt: 2019.2.0

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: Not Installed
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 3.6.6 (default, Mar 29 2019, 00:03:27)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.3.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.4.2
            ZMQ: 4.1.4

System Versions:
           dist: centos 7.6.1810 Core
         locale: UTF-8
        machine: x86_64
        release: 3.10.0-957.1.3.el7.x86_64
         system: Linux
        version: CentOS Linux 7.6.1810 Core

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 3
Comments: 32 (18 by maintainers)

Links to this issue

Unable to use modules in custom grains

Most upvoted comments

@lorengordon FYI with the next release, shall look at not using the Redhat recommended macro py3_install and instead different command to perform the Py 3 install and lose the ‘-s’. The next release should be soon too.

dmurphy18 on Nov 7, 2019

@pstengel There have been requests that SaltStack package pygit2 and its associated libgit2 in the past and they have been rejected due to nature of pygit2 and libgit2 versions having to be tied together, and how that can be comprised when updates are made to pygit2 and libgit2 outside of the versions that SaltStack would provide.

As for removing -s, I cannot comment on that.

dmurphy18 on Apr 19, 2019

@wb529kq

I get a python script:

#!/usr/bin/env python

# Python Libs
import os
import subprocess


SALT_BIN_PATH = "/usr/bin"
FIRST_LINE = "#!/usr/bin/python3"
SERVICES_TO_RESTART = ['salt-master', 'salt-minion']


files_to_patch = []
for file in os.listdir(SALT_BIN_PATH):
    if file.startswith('salt'):
        files_to_patch.append(os.path.join(SALT_BIN_PATH, file))

for file_to_patch in files_to_patch:
    print('patch : {}'.format(file_to_patch))
    new_file = [FIRST_LINE]
    with open(file_to_patch, 'r') as file:
        file_content = file.read().split(os.linesep)
        file_content.pop(0)
        new_file.extend(file_content)
    with open(file_to_patch, 'w') as file:
        file.write(os.linesep.join(new_file))

for service in SERVICES_TO_RESTART:
    try:
        print('restarting {} service'.format(service))
        subprocess.check_call(['systemctl', 'restart', service])
    except subprocess.CalledProcessError as e:
        ploter(messages=["Commande    : {}".format(e.cmd)], level='ERROR')
        ploter(messages=["Return code : {}".format(e.returncode)], level='ERROR')
        ploter(messages=["OutPut      : {}".format(e.output)], level='ERROR')
        exit(1)

print("Your salt bin was updated, and services restarted.")

On the cluster run:

sudo python <correct_install>.py

ArthurREGNARD on Jan 24, 2020