salt: [BUG] salt-ssh jinja import from doesn't work due to error jinja2.exceptions.TemplateNotFound

Description Jinja import from doesn’t work with salt-ssh due to error jinja2.exceptions.TemplateNotFound. It works fine while running same setup via salt-minion, so problem is probably in salt-ssh.

Debug output:

[DEBUG   ] In saltenv 'base', looking at rel_path 'salt/repo.sls' to resolve 'salt://salt/repo.sls'
[DEBUG   ] In saltenv 'base', ** considering ** path '/var/tmp/.root_29fa9c_salt/running_data/var/cache/salt/minion/files/base/salt/repo.sls' to resolve 'salt://salt/repo.sls'
[DEBUG   ] compile template: /var/tmp/.root_29fa9c_salt/running_data/var/cache/salt/minion/files/base/salt/repo.sls
[DEBUG   ] Jinja search path: ['/var/tmp/.root_29fa9c_salt/running_data/var/cache/salt/minion/files/base']
[DEBUG   ] In saltenv 'base', looking at rel_path 'salt/map.jinja' to resolve 'salt://salt/map.jinja'
[DEBUG   ] In saltenv 'base', ** considering ** path '/var/cache/salt/master/files/base/salt/map.jinja' to resolve 'salt://salt/map.jinja'
[DEBUG   ] Fetching file from saltenv 'base', ** attempting ** 'salt://salt/map.jinja'
[DEBUG   ] No dest file found
[INFO    ] Fetching file from saltenv 'base', ** done ** 'salt/map.jinja'
[DEBUG   ] Jinja Error
[DEBUG   ] Exception:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 502, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 37, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 1, in top-level template code
  File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 198, in get_source
    raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: salt/map.jinja
[DEBUG   ] Out:
[DEBUG   ] Line: 1
[DEBUG   ] TmplStr: {% from "salt/map.jinja" import salt with context %}

But file /var/cache/salt/master/files/base/salt/map.jinja is present in salt master FS.

Setup Install salt-master + salt-ssh and setup roster file. The problem occurs at Debian 9 VM with salt v3004. Tested with salt v 2019.2.8 without any problem. No minion installed at target machine, target machine is clean Debian installation.

Steps to Reproduce the behavior Run salt-ssh \* state.sls test with these states: test/init.sls:

include:
  - .repo

...

test/repo.sls:

{% from "test/map.jinja" import salt with context %}

...

test/map.jinja:

{% set salt = salt['grains.filter_by']({
  'stretch': {
    'repo': 'deb [arch=amd64] https://repo.saltproject.io/py3/debian/9/amd64/3004 stretch main',
    'key_url': 'https://repo.saltproject.io/py3/debian/9/amd64/3004/salt-archive-keyring.gpg'
  },
  'buster': {
    'repo': 'deb [arch=amd64] https://repo.saltproject.io/py3/debian/10/amd64/3004 buster main',
    'key_url': 'https://repo.saltproject.io/py3/debian/10/amd64/3004/salt-archive-keyring.gpg'
  },
}, 'lsb_distrib_codename') %}

Expected behavior No error and success as running salt via salt-minion.

Versions Report

salt --versions-report ``` Salt Version: Salt: 3004

Dependency Versions: cffi: Not Installed cherrypy: Not Installed dateutil: 2.5.3 docker-py: Not Installed gitdb: 2.0.0 gitpython: 2.1.1 Jinja2: 2.9.4 libgit2: Not Installed M2Crypto: Not Installed Mako: Not Installed msgpack: 0.6.2 msgpack-pure: Not Installed mysql-python: Not Installed pycparser: Not Installed pycrypto: Not Installed pycryptodome: 3.6.1 pygit2: Not Installed Python: 3.5.3 (default, Apr 5 2021, 09:00:41) python-gnupg: Not Installed PyYAML: 3.12 PyZMQ: 17.1.2 smmap: 2.0.1 timelib: Not Installed Tornado: 4.5.3 ZMQ: 4.2.1

System Versions: dist: debian 9 stretch locale: UTF-8 machine: x86_64 release: 4.9.0-16-amd64 system: Linux version: Debian GNU/Linux 9 stretch

</details>

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Comments: 21 (7 by maintainers)

Most upvoted comments

@Rudd-O yeah, we just finished a migration from ansible to salt-ssh, just in time for all these 3003 bugs. it’s been a real headache. def feels like salt-ssh isn’t getting the regression testing / QA / dev love as the rest of the stack, which is really unfortunate.

Didn’t notice 3007 was release, generally I wait .1 or .2 before upgrading. I had too much issue in the past with upgrading. I will try to test on my labs

so when is 3006 getting released?

Still getting this with a 3005.1.4 master when I attempt a relative jinja import (from './vars.jinja' import...).

I believe the fix is #63184 which is not in 3005.1.4 but scheduled for 3006

I’m running into this issue now with 3004 as well. I’ve never had to use extra_filerefs previously, but now it seems busted. I’ve noticed that, without adding extra_filerefs, if I call a specific state that references the include, it works. but if I call the highstate (such as with salt-ssh hostname state.apply) then it breaks as mentioned here.

is there a workaround that actually works for this case? and is the root cause potentially the cache dir issue that seems to be causing #60003?

I just upgraded our master from 3002.7 to 3004 and got same error calling state over salt-ssh like, all formulas worked ok when called from master->minion or on minion via salt-call with no problem.

SSH command:

salt-ssh -v "*" state.sls prometheus

and i was getting error from all sls files:

    - Rendering SLS 'base:prometheus.node' failed: Jinja error: prometheus/map.jinja
    ...
      {% from "prometheus/map.jinja" import binExporter with context %}    <======================

I did try adding --extra-filerefs, but it did not helped. Since i have under this prometheus defined all kind of services, i wanted to lower the error output i was getting and called

salt-ssh -v "*" state.sls prometheus.nginx

To my suprise did get exececuted without problem and after that full formula works as well. The only thing maybe some corrupted cache after upgrade?