salt: [BUG] gitfs is suddenly a hassle with onedir package

Description I upgraded to the 3005 onedir package on my salt master. The salt-master is configured to use gitfs and git_pillar via pygit2. I expected this critical salt master functionality to “just work” but I actually had to install the pygit2 module using salt pip install pygit2 --no-deps. I used no-deps as pip drags in a newer version of cffi conflicting with the included onedir version if I don’t.

However, pygit2 fails during startup of the salt master because ssl setup fails. I only use ssh git urls, but being able to configure ca certificates location seems to be a requirement for the pygit2 module.

During startup the salt master fails with:

2022-09-08 08:34:15,880 [salt.utils.gitfs :148 ][ERROR   ][1607008] Failed to import pygit2
Traceback (most recent call last):
  File "salt/utils/gitfs.py", line 108, in <module>
    import pygit2
  File "/opt/saltstack/salt/pypath/lib/python3.9/site-packages/pygit2/__init__.py", line 230, in <module>
    settings = Settings()
  File "/opt/saltstack/salt/pypath/lib/python3.9/site-packages/pygit2/settings.py", line 55, in __init__
    self._initialize_tls_certificate_locations()
  File "/opt/saltstack/salt/pypath/lib/python3.9/site-packages/pygit2/settings.py", line 61, in _initialize_tls_certificate_l
ocations
    self.set_ssl_cert_locations(
  File "/opt/saltstack/salt/pypath/lib/python3.9/site-packages/pygit2/settings.py", line 191, in set_ssl_cert_locations
    option(_pygit2.GIT_OPT_SET_SSL_CERT_LOCATIONS, cert_file, cert_dir)
_pygit2.GitError: OpenSSL error: failed to load certificates: error:00000000:lib(0):func(0):reason(0)
2022-09-08 08:34:16,151 [salt.master      :2475][INFO    ][1602761] User root Published command test.version with jid 2022090
8063416149168

I found that I could solve this by setting the SSL_CERT_FILE environment variable to the os default location which is /etc/ssl/certs/ca-certificates.crt in my case. I could not find a way to configure this in the configuration so I have to do it the cumbersome way by making an systemd override file /etc/systemd/system/salt-master.service.d/override.conf containing:

[Service]
Environment=SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt

This fixes the master startup error, and minions can connect and execute states afterwards. However, for some reason if I execute salt-call for the local master minion I still get the pygit2 ssl cert error printed to the screen, while the states executes seemingly properly. To fix this I have to ensure the SSL_CERT_FILE env var is set for manual salt-call invokations of the minion on the master itself e.g:

sudo SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt salt-call state.apply

This is obviously cumbersome, so I would like a way for this to be configured once only, prefererrably within saltstack, not using linux workarounds like systemd overrides and shell aliases.

• Also: pygit2 version is not reported when printing the version report even though it is available within the salt onedir env.

Steps to Reproduce the behavior

• Install onedir package
• Install missing pygit2 dependency in salt onedir env: salt pip install pygit2 --no-deps
• Install missing libgit2 dependency in OS (apt install libgit2-1.1 in Debian 11)
• Configure gitfs_provider as pygit2 with valid gitfs configuration and repo
• Try to start salt master

Expected behavior

• Minimum: document how to use gitfs properly with onedir
• Expected: pygit2 should be included in onedir package and SSL_CERT_DIR/SSL_CERT_FILE env params should be configurable globally in the salt master config or automatically set to a sane default.

Versions Report

Salt Version:
          Salt: 3005
 
Dependency Versions:
          cffi: 1.14.6
      cherrypy: unknown
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.0
       libgit2: Not Installed
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.9.8
        pygit2: Not Installed
        Python: 3.9.13 (main, Aug 23 2022, 18:31:09)
  python-gnupg: 0.4.8
        PyYAML: 5.4.1
         PyZMQ: 23.2.0
         smmap: Not Installed
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.4
 
System Versions:
          dist: debian 11 bullseye
        locale: utf-8
       machine: x86_64
       release: 5.10.0-17-amd64
        system: Linux
       version: Debian GNU/Linux 11 bullseye

# sudo salt pip freeze
pygit2==1.10.1