salt: [BUG] 3006.5 "grains" extmods folder cleaned in unexpected situations
Description In 3006.5, the extmods grains folder is removed in many situations:
when the --local option is passed to a salt-call command and the minion has previously sync’d grains from a master, losing all previously synchronized grain modules
The folder also seems to be removed if the minion cannot connect to the master. I.E. a mastered minion, whose master is down, will clean its grains folder and lose all sync’d grains until the master is “alive” again and they can be resync’d.
The folder is removed on other commands, even when the master is alive, for example, salt-call test.ping will remove the grains extmods folder and the minion will lose sync’d grains.
Setup (Please provide relevant configs and/or SLS files (be sure to remove sensitive info. There is no general set-up of Salt.)
Please be as specific as possible and give set-up details.
- on-prem machine
- VM (Virtualbox, KVM, etc. please specify)
- VM running on a cloud service, please be explicit and add details
- container (Kubernetes, Docker, containerd, etc. please specify)
- or a combination, please be explicit
- jails if it is FreeBSD
- classic packaging
- onedir packaging
- used bootstrap to install
Steps to Reproduce the behavior
Situation 1: “–local” option
configure a minion with a master that provides additional grain modules via the _grains folder
sync the grains from the master salt-call saltutil.sync_grains or salt-call saltutil.sync_all, /var/cache/salt/minion/grains should have the grains modules from the _grains folder on the master
use salt-call --local <anything> and the /var/cache/salt/minion/grains folder will be removed - other extmods folders (like modules, states, etc do not seem to be affected).
Situation 2: module commands:
salt-call saltutil.sync_all extmods grains folder exists
salt-call test.ping grains extmods folder is removed (other modules tested where extmods grains folder is removed: grains.get, grains.items
Situation 3: master is down
salt-call saltutil.sync_all - extmods grains folder exists
…take master offline…
salt-call grains.items - extmods grains folder is removed
Expected behavior
The grains extmods folder is not cleaned in any of the specified scenarios.
Screenshots If applicable, add screenshots to help explain your problem.
Versions Report
salt --versions-report
(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)Salt Version:
Salt: 3006.5
Python Version:
Python: 3.10.13 (main, Nov 15 2023, 04:34:27) [GCC 11.2.0]
Dependency Versions:
cffi: 1.14.6
cherrypy: 18.6.1
dateutil: 2.8.1
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 3.1.2
libgit2: Not Installed
looseversion: 1.0.2
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 1.0.2
msgpack-pure: Not Installed
mysql-python: Not Installed
packaging: 22.0
pycparser: 2.21
pycrypto: Not Installed
pycryptodome: 3.9.8
pygit2: Not Installed
python-gnupg: 0.4.8
PyYAML: 6.0.1
PyZMQ: 23.2.0
relenv: 0.14.2
smmap: Not Installed
timelib: 0.2.4
Tornado: 4.5.3
ZMQ: 4.3.4
System Versions:
dist: rhel 9.3 Plow
locale: utf-8
machine: x86_64
release: 5.14.0-362.13.1.el9_3.x86_64
system: Linux
version: Red Hat Enterprise Linux 9.3 Plow
Additional context Based on 3006.5 release notes alone, I would suspect this is a side effect of #65186
A minion that is explicitly configured as “local” does not seem to exhibit any of these symptoms.
About this issue
- Original URL
- State: closed
- Created 7 months ago
- Reactions: 4
- Comments: 30 (27 by maintainers)
Commits related to this issue
- pin salt 3006.4 until we have a fix for https://github.com/saltstack/salt/issues/65692 (i.e. a release with https://github.com/saltstack/salt/pull/65738) — committed to jdelic/saltshaker by jdelic 6 months ago
- Revert ssalt to 3006.4 https://github.com/saltstack/salt/issues/65692 temporarily until the fix for this bug is released in 3006.7 — committed to scality/metalk8s by eg-ayoub 4 months ago
- Revert ssalt to 3006.4 https://github.com/saltstack/salt/issues/65692 temporarily until the fix for this bug is released in 3006.7 — committed to scality/metalk8s by eg-ayoub 4 months ago
- Revert ssalt to 3006.4 https://github.com/saltstack/salt/issues/65692 temporarily until the fix for this bug is released in 3006.7 — committed to scality/metalk8s by eg-ayoub 4 months ago
- Revert ssalt to 3006.4 https://github.com/saltstack/salt/issues/65692 temporarily until the fix for this bug is released in 3006.7 — committed to scality/metalk8s by eg-ayoub 4 months ago
A patch for 3006.5/6:
65738.diff
Reverting changes in PR https://github.com/saltstack/salt/pull/65186, since incomplete solution, will redo in another PR later, but want to quickly fix the issue, and will address the original issue #65027 with better testing etc.
the breakage in Salt 3006.5 has been rolled back and normal grains should be available in the next bug release which will be soon, had to get the CVE release out first