salt: Order in top.sls file is not respected for pillar data in local mode

My pillar setup looks like this:

In the minion.conf the pillar_roots config looks like this:

pillar_roots:
  lib:
    - /srv/salt/salt-lib/pillar
  base:
    - /srv/salt/pillar

My pillar top.sls file in /srv/salt/pillar/top.sls looks like this:

lib:
  '*':
    - main

base:
  '*':
    - basic

The main.sls file in /srv/salt/salt-lib/pillar/ contains:

test:
  setting: myvalue-main

The basic.sls file in /srv/salt/pillar/ contains:

test:
  setting: myvalue-override

When I now run

sudo salt-call --local pillar.get test:setting

the output is:

local:
    myvalue-main

The expected output would be:

local:
    myvalue-override

The salt version is 2014.7.1

Is this related to #1432 ?

About this issue

Original URL
State: closed
Created 9 years ago
Reactions: 2
Comments: 25 (16 by maintainers)

Commits related to this issue

Preserve order of pillar_roots in _get_envs() _get_envs() returns pillar_roots in a non-deterministic order. Update _get_envs() to preserve the order of pillar_roots similar to #e36d683 for file_root... — committed to saltstack/salt by mkirkland4874 3 years ago
Add changelog for #24501 — committed to saltstack/salt by mkirkland4874 3 years ago
Add regression tests for #24501 — committed to saltstack/salt by s0undt3ch 3 years ago

Most upvoted comments

Any idea when this will get fixed? It was marked bug, high severity 2.5 years ago. It would immensely simplify many of my pillars if this worked.

wwalker on Apr 23, 2018

@jfindlay I tried the settings pillar_source_merging_strategy with the default smart setting and the overwrite setting. The result is still myvalue-main.

I looked at 2015.5.5 and 2015.8.0 and the result is the same.

The source of the problem seems to be preserving the order of environments in pillar/__init__.py: It starts out with self._get_envs() returning a standard set of environments defined as the pillars in pillar_root, and continues throughout the file with collections.defaultdict, for example in merge_tops(). While the pillars within each environment are ordered, the environments themselves are not ordered.

The behaviour I would expect: A deterministic order of the top files to be read (as listed in pillar_roots?), and then processing the environments as listed in each top file.

My workaround is now to have an external pillar to override values, as all external pillars override pillar_roots as a default (unless ext_pillar_first is specified). Any external pillar will do:

ext_pillar:
  - cmd_yaml:
      command: cat /srv/salt/pillar/basic.sls

ahus1 on Oct 3, 2015