salt: States do not fail if ext_pillar does not provide any data

Expected Behavior

If an ext_pillar module does not provide any data a (high-)state should fail. The problem is if we need pillar data from foreman but it doesn’t provide any data a minion could go into an unusable state.

e.g. We define interface information in foreman and a highstate is triggered on a minion, but foreman is currently not available - all interfaces on a minion would be removed.

Actual Behavior

States do not fail if an ext_pillar module doesn’t provide any data.

Versions Report

Salt Version:
           Salt: 2015.8.5

Dependency Versions:
         Jinja2: 2.7.2
       M2Crypto: Not Installed
           Mako: 0.9.1
         PyYAML: 3.10
          PyZMQ: 14.0.1
         Python: 2.7.6 (default, Jun 22 2015, 17:58:13)
           RAET: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.4
           cffi: 1.5.0
       cherrypy: 3.2.2
       dateutil: 1.5
          gitdb: 0.5.4
      gitpython: 0.3.2 RC1
          ioflo: Not Installed
        libgit2: Not Installed
        libnacl: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.3.0
   mysql-python: 1.2.3
      pycparser: 2.14
       pycrypto: 2.6.1
         pygit2: Not Installed
   python-gnupg: Not Installed
          smmap: 0.8.2
        timelib: Not Installed

System Versions:
           dist: Ubuntu 14.04 trusty
        machine: x86_64
        release: 3.13.0-77-generic
         system: Ubuntu 14.04 trusty

Workaround

We created a state[1] which checks for a pillar (pillar_available) which is defined in foreman and will be run first. If that pillar key is not available the highstate will fail hard.

[1]

check_foreman_pillar_available:
  test.check_pillar:
    - boolean: pillar_available
    - order: 1
    - failhard: True

About this issue

Original URL
State: closed
Created 8 years ago
Comments: 15 (14 by maintainers)

Commits related to this issue

update how we install saltstack on the base AMI This is based on http://repo.saltstack.com/#ubuntu, which is the "official" install - the ppa used previously eventually became the "commumnity support... — committed to fpco/terraform-aws-foundation by ketzacoatl 8 years ago
import pip state and module from 2015.8, for use with 2015.5 this is meant to be a temporary step until we are able to use a version of salt that is more recent than 2015.5.x. See this issue for more... — committed to fpco/fpco-salt-formula by ketzacoatl 8 years ago
update how we install saltstack on the base AMI This is based on http://repo.saltstack.com/#ubuntu, which is the "official" install - the ppa used previously eventually became the "commumnity support... — committed to fpco/terraform-aws-foundation by ketzacoatl 8 years ago

Most upvoted comments

To clarify, and to ensure we’re on the same page… the traceback I posted above is “consul is not available”, and the ext_pillar (and pillar) module continue on (soft fail), while the logs I posted before that are “consul is not available and everything blows up” (hard fail). The soft fail is from older versions of salt, while the hard failure is from the more recent release (2016.x).

I’m ok with the soft fail (especially b/c I can use the assertions you pointed out in the other ticket), but it’s the hard fail that is causing me serious headache. Lastly, the config option I am referring to would tell salt to let the ext_pillar fail without stopping the state.sls or whatever (eg, allow a soft fail).

ketzacoatl on Jun 3, 2016