salt: [BUG] Jinja2 3.0 support breaking changes

Jinja2 3.0 was released a few days ago and seems like Salt can’t work with it.

We have issues like this:

    Data failed to compile:'
----------'
    Rendering SLS ''base:formula.logstash.common/install'' failed: Jinja variable ''pillar'' is undefined'
/opt/behavox/salt/cache/minion/files/base/formula/kibana/vars.yaml(2):'
---'
---'
{% set third_party_packages_path = pillar.get(''third_party_packages_path'') %}    <======================'
[...]'

About this issue

Original URL
State: closed
Created 3 years ago
Reactions: 1
Comments: 15 (9 by maintainers)

Commits related to this issue

feat(master): use Jinja2 downgrade workaround (`3.0.x` => `2.11.3`) Various discussions about the underlying issue: * https://github.com/saltstack/salt/issues/60188 * https://github.com/saltstack-fo... — committed to netmanagers/salt-image-builder by myii 3 years ago
feat(packages_versions): add/update text files for FreeBSD [skip ci] Note, `Jinja2` wasn't downgraded, as per the upstream issue (https://github.com/saltstack/salt/issues/60188) -- appears that this ... — committed to netmanagers/salt-image-builder by myii 3 years ago
fix(map): workaround Jinja2 3.0 bug A change in Jinja2 3.0 cause troubles in rendering templates where salt special variables are available only on first import but not subsequent ones (see https://g... — committed to eole/template-formula by baby-gnu 3 years ago
fix(map): workaround salt bug with Jinja2 3.0 A change in Jinja2 3.0 cause troubles in rendering templates where salt special variables are available only on first import but not subsequent ones (see... — committed to eole/template-formula by baby-gnu 3 years ago
test(jinja): add state files for testing known Jinja issues Check for problems associated with this bug: * https://github.com/saltstack/salt/issues/60188 - Affects use of `grains`, `opts`, `pillar... — committed to netmanagers/salt-image-builder by myii 2 years ago

Most upvoted comments

As mentioned in Slack:

I’m wondering if this should actually be considered to be a bug rather than a feature request. We’ve started seeing failures across various formulas and I did a test run to determine that 32 of around 100 formulas are now failing: https://saltstack-formulas.zulipchat.com/#narrow/stream/239693-CI/topic/myii-ci.2F2021-W20d/search/fail.

Here’s an example of a PR that fixes the issue: https://github.com/saltstack-formulas/postgres-formula/pull/311. I’m not so sure this will work everywhere, though.

Should we patch our 32 formulas or should we wait for a resolution in the main Salt repo?

myii on May 18, 2021

Now affecting openSUSE Tumbleweed stable builds with the salt-bootstrap. That’s alongside Arch LInux stable builds and most/all of the git builds.

And now also affects our newly built pre-salted master FreeBSD Vagrant boxes:

Update: I’ve uploaded version 2021.07.22 for all three above that now includes the Jinja2 downgrade workaround (which is actually working with our formulas again). So if anyone is checking out these boxes, the 2021.07.21 version is the one with Jinja2 3.0.1.

myii on Jul 22, 2021

Now affecting openSUSE Tumbleweed stable builds with the salt-bootstrap. That’s alongside Arch Linux stable builds and most/all of the git builds.

myii on Jul 27, 2021