salt: [BUG] salt-ssh jinja import from doesn't work due to error jinja2.exceptions.TemplateNotFound
Description Jinja import from doesn’t work with salt-ssh due to error jinja2.exceptions.TemplateNotFound. It works fine while running same setup via salt-minion, so problem is probably in salt-ssh.
Debug output:
[DEBUG ] In saltenv 'base', looking at rel_path 'salt/repo.sls' to resolve 'salt://salt/repo.sls'
[DEBUG ] In saltenv 'base', ** considering ** path '/var/tmp/.root_29fa9c_salt/running_data/var/cache/salt/minion/files/base/salt/repo.sls' to resolve 'salt://salt/repo.sls'
[DEBUG ] compile template: /var/tmp/.root_29fa9c_salt/running_data/var/cache/salt/minion/files/base/salt/repo.sls
[DEBUG ] Jinja search path: ['/var/tmp/.root_29fa9c_salt/running_data/var/cache/salt/minion/files/base']
[DEBUG ] In saltenv 'base', looking at rel_path 'salt/map.jinja' to resolve 'salt://salt/map.jinja'
[DEBUG ] In saltenv 'base', ** considering ** path '/var/cache/salt/master/files/base/salt/map.jinja' to resolve 'salt://salt/map.jinja'
[DEBUG ] Fetching file from saltenv 'base', ** attempting ** 'salt://salt/map.jinja'
[DEBUG ] No dest file found
[INFO ] Fetching file from saltenv 'base', ** done ** 'salt/map.jinja'
[DEBUG ] Jinja Error
[DEBUG ] Exception:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
output = template.render(**decoded_context)
File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "<template>", line 1, in top-level template code
File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: salt/map.jinja
[DEBUG ] Out:
[DEBUG ] Line: 1
[DEBUG ] TmplStr: {% from "salt/map.jinja" import salt with context %}
But file /var/cache/salt/master/files/base/salt/map.jinja is present in salt master FS.
Setup Install salt-master + salt-ssh and setup roster file. The problem occurs at Debian 9 VM with salt v3004. Tested with salt v 2019.2.8 without any problem. No minion installed at target machine, target machine is clean Debian installation.
Steps to Reproduce the behavior
Run salt-ssh \* state.sls test with these states:
test/init.sls:
include:
- .repo
...
test/repo.sls:
{% from "test/map.jinja" import salt with context %}
...
test/map.jinja:
{% set salt = salt['grains.filter_by']({
'stretch': {
'repo': 'deb [arch=amd64] https://repo.saltproject.io/py3/debian/9/amd64/3004 stretch main',
'key_url': 'https://repo.saltproject.io/py3/debian/9/amd64/3004/salt-archive-keyring.gpg'
},
'buster': {
'repo': 'deb [arch=amd64] https://repo.saltproject.io/py3/debian/10/amd64/3004 buster main',
'key_url': 'https://repo.saltproject.io/py3/debian/10/amd64/3004/salt-archive-keyring.gpg'
},
}, 'lsb_distrib_codename') %}
Expected behavior No error and success as running salt via salt-minion.
Versions Report
salt --versions-report
``` Salt Version: Salt: 3004Dependency Versions: cffi: Not Installed cherrypy: Not Installed dateutil: 2.5.3 docker-py: Not Installed gitdb: 2.0.0 gitpython: 2.1.1 Jinja2: 2.9.4 libgit2: Not Installed M2Crypto: Not Installed Mako: Not Installed msgpack: 0.6.2 msgpack-pure: Not Installed mysql-python: Not Installed pycparser: Not Installed pycrypto: Not Installed pycryptodome: 3.6.1 pygit2: Not Installed Python: 3.5.3 (default, Apr 5 2021, 09:00:41) python-gnupg: Not Installed PyYAML: 3.12 PyZMQ: 17.1.2 smmap: 2.0.1 timelib: Not Installed Tornado: 4.5.3 ZMQ: 4.2.1
System Versions: dist: debian 9 stretch locale: UTF-8 machine: x86_64 release: 4.9.0-16-amd64 system: Linux version: Debian GNU/Linux 9 stretch
</details>
About this issue
- Original URL
- State: open
- Created 3 years ago
- Comments: 21 (7 by maintainers)
Hello.
I made some tests and found it broke at 3003.
Regards.
@Rudd-O https://docs.saltproject.io/salt/install-guide/en/latest/topics/upgrade-to-onedir.html#upgrade-to-onedir
@Rudd-O yeah, we just finished a migration from ansible to salt-ssh, just in time for all these
3003bugs. it’s been a real headache. def feels likesalt-sshisn’t getting the regression testing / QA / dev love as the rest of the stack, which is really unfortunate.Didn’t notice 3007 was release, generally I wait .1 or .2 before upgrading. I had too much issue in the past with upgrading. I will try to test on my labs
so when is 3006 getting released?
I believe the fix is #63184 which is not in
3005.1.4but scheduled for3006I’m running into this issue now with
3004as well. I’ve never had to useextra_filerefspreviously, but now it seems busted. I’ve noticed that, without addingextra_filerefs, if I call a specific state that references the include, it works. but if I call the highstate (such as withsalt-ssh hostname state.apply) then it breaks as mentioned here.is there a workaround that actually works for this case? and is the root cause potentially the cache dir issue that seems to be causing #60003?
I just upgraded our master from 3002.7 to 3004 and got same error calling state over salt-ssh like, all formulas worked ok when called from master->minion or on minion via salt-call with no problem.
SSH command:
and i was getting error from all sls files:
I did try adding
--extra-filerefs, but it did not helped. Since i have under this prometheus defined all kind of services, i wanted to lower the error output i was getting and calledTo my suprise did get exececuted without problem and after that full formula works as well. The only thing maybe some corrupted cache after upgrade?