salt: grains.filter_by grain value doesn't support list

Description of Issue/Question

grains.filter_by grain value doesn’t support a list when use list ,return AttributeError: ‘list’ object has no attribute ‘split’

Setup

map.jinja:

{% set fluentd = salt['grains.filter_by']({
    'Ubuntu-12.04': {'pkg': 'td-agent_2.3.4-0.precise.amd64.deb'},
    'Ubuntu-14.04': {'pkg': 'td-agent_2.3.4-0.precise.amd64.deb'},
    'Ubuntu-16.04': {'pkg': 'td-agent_2.3.4-0.precise.amd64.deb'},
    'CentOS-6': {'pkg': 'td-agent_2.3.4-0.el6.amd64.deb'},
    'CentOS-7': {'pkg': 'td-agent_2.3.4-0.precise.amd64.deb'},
    'Debian': {'pkg': 'td-agent_2.3.4-0.precise.amd64.deb'},
    'RedHat': {'pkg': 'td-agent_2.3.4-0.elrh.amd64.deb'},
},grain=['osfinger','os_family'], default='RedHat') %}

init.sls:

{% from "install/fluentd/map.jinja" import fluentd with context %}
/tmp/salt.debug:
  file.managed:
    - contents: |
    {%- for k,v in fluentd.items() %}
        {{ k }} => {{ v }}
    {%- endfor %}

Steps to Reproduce Issue

salt -S 192.168.12.192 state.sls install.fluentd

192.168.12.192:
    Data failed to compile:
----------
    Rendering SLS 'base:install.fluentd' failed: Jinja error: 'list' object has no attribute 'split'
/var/cache/salt/minion/files/base/install/fluentd/map.jinja(1):
---
{% set fluentd = salt['grains.filter_by']({    <======================
    'Ubuntu-12.04': {'pkg': 'td-agent_2.3.4-0.precise.amd64.deb'},
    'Ubuntu-14.04': {'pkg': 'td-agent_2.3.4-0.precise.amd64.deb'},
    'Ubuntu-16.04': {'pkg': 'td-agent_2.3.4-0.precise.amd64.deb'},
    'CentOS-6': {'pkg': 'td-agent_2.3.4-0.el6.amd64.deb'},
    'CentOS-7': {'pkg': 'td-agent_2.3.4-0.precise.amd64.deb'},
[...]
---
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/salt/utils/templates.py", line 368, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib64/python2.6/site-packages/jinja2/environment.py", line 669, in render
    return self.environment.handle_exception(exc_info, True)
  File "<template>", line 1, in top-level template code
  File "/usr/lib64/python2.6/site-packages/jinja2/environment.py", line 713, in make_module
    return TemplateModule(self, self.new_context(vars, shared, locals))
  File "/usr/lib64/python2.6/site-packages/jinja2/environment.py", line 769, in __init__
    self._body_stream = list(template.root_render_func(context))
  File "/var/cache/salt/minion/files/base/install/fluentd/map.jinja", line 1, in top-level template code
    {% set fluentd = salt['grains.filter_by']({
  File "/usr/lib/python2.6/site-packages/salt/modules/grains.py", line 555, in filter_by
    val = salt.utils.traverse_dict_and_list(__grains__, grain, [])
  File "/usr/lib/python2.6/site-packages/salt/utils/__init__.py", line 1547, in traverse_dict_and_list
    for each in key.split(delimiter):
AttributeError: 'list' object has no attribute 'split'

Versions Report

Salt Version:
           Salt: 2016.11.2
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 1.5
          gitdb: 0.5.4
      gitpython: 0.3.2 RC1
          ioflo: Not Installed
         Jinja2: 2.7.2
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 0.9.1
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: 1.2.3
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
         Python: 2.7.6 (default, Jun 22 2015, 17:58:13)
   python-gnupg: Not Installed
         PyYAML: 3.10
          PyZMQ: 14.0.1
           RAET: Not Installed
          smmap: 0.8.2
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5
 
System Versions:
           dist: Ubuntu 14.04 trusty
        machine: x86_64
        release: 3.19.0-25-generic
         system: Linux
        version: Ubuntu 14.04 trusty

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Comments: 22 (8 by maintainers)

Most upvoted comments

Still relevant. I can replicate it as follows:

test_grains.jinja

{% set test_grains = salt['grains.filter_by']({
    'Ubuntu-16.04': {
        'apacheuser': 'www-data',
    },
    'Ubuntu-18.04': {
        'apacheuser': 'www-data',
    },
    'RedHat': {
        'apacheuser': 'apache',
    },
}, grain=['os_finger', 'osfamily']) %}

test_grains.sls

{% from "test_grains.jinja" import test_grains with context %}
echo {{ test_grains.apacheuser }}:
  cmd.run
# salt-call --local --file-root=./states/ state.sls test_grains test=True
[ERROR   ] Rendering exception occurred
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/utils/templates.py", line 169, in render_tmpl
    output = render_str(tmplstr, context, tmplpath)
  File "/usr/lib/python2.7/dist-packages/salt/utils/templates.py", line 443, in render_jinja_tmpl
    trace=tracestr)
SaltRenderError: Jinja error: 'list' object has no attribute 'split'
/var/cache/salt/minion/files/base/test_grains.jinja(1):
---
{% set test_grains = salt['grains.filter_by']({    <======================
    'Ubuntu-16.04': {
        'apacheuser': 'www-data',
    },
    'Ubuntu-18.04': {
        'apacheuser': 'www-data',
[...]
---
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/utils/templates.py", line 394, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "<template>", line 1, in top-level template code
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 1073, in make_module
    return TemplateModule(self, self.new_context(vars, shared, locals))
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 1152, in __init__
    body_stream = list(template.root_render_func(context))
  File "/var/cache/salt/minion/files/base/test_grains.jinja", line 1, in top-level template code
    {% set test_grains = salt['grains.filter_by']({
  File "/usr/lib/python2.7/dist-packages/salt/modules/grains.py", line 600, in filter_by
    base=base)
  File "/usr/lib/python2.7/dist-packages/salt/utils/data.py", line 504, in filter_by
    val = traverse_dict_and_list(traverse, lookup, [])
  File "/usr/lib/python2.7/dist-packages/salt/utils/data.py", line 578, in traverse_dict_and_list
    for each in key.split(delimiter):
AttributeError: 'list' object has no attribute 'split'


[CRITICAL] Rendering SLS 'base:test_grains' failed: Jinja error: 'list' object has no attribute 'split'
/var/cache/salt/minion/files/base/test_grains.jinja(1):
---
{% set test_grains = salt['grains.filter_by']({    <======================
    'Ubuntu-16.04': {
        'apacheuser': 'www-data',
    },
    'Ubuntu-18.04': {
        'apacheuser': 'www-data',
[...]
---
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/utils/templates.py", line 394, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "<template>", line 1, in top-level template code
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 1073, in make_module
    return TemplateModule(self, self.new_context(vars, shared, locals))
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 1152, in __init__
    body_stream = list(template.root_render_func(context))
  File "/var/cache/salt/minion/files/base/test_grains.jinja", line 1, in top-level template code
    {% set test_grains = salt['grains.filter_by']({
  File "/usr/lib/python2.7/dist-packages/salt/modules/grains.py", line 600, in filter_by
    base=base)
  File "/usr/lib/python2.7/dist-packages/salt/utils/data.py", line 504, in filter_by
    val = traverse_dict_and_list(traverse, lookup, [])
  File "/usr/lib/python2.7/dist-packages/salt/utils/data.py", line 578, in traverse_dict_and_list
    for each in key.split(delimiter):
AttributeError: 'list' object has no attribute 'split'


local:
    Data failed to compile:
----------
    Rendering SLS 'base:test_grains' failed: Jinja error: 'list' object has no attribute 'split'
/var/cache/salt/minion/files/base/test_grains.jinja(1):
---
{% set test_grains = salt['grains.filter_by']({    <======================
    'Ubuntu-16.04': {
        'apacheuser': 'www-data',
    },
    'Ubuntu-18.04': {
        'apacheuser': 'www-data',
[...]
---
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/utils/templates.py", line 394, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "<template>", line 1, in top-level template code
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 1073, in make_module
    return TemplateModule(self, self.new_context(vars, shared, locals))
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 1152, in __init__
    body_stream = list(template.root_render_func(context))
  File "/var/cache/salt/minion/files/base/test_grains.jinja", line 1, in top-level template code
    {% set test_grains = salt['grains.filter_by']({
  File "/usr/lib/python2.7/dist-packages/salt/modules/grains.py", line 600, in filter_by
    base=base)
  File "/usr/lib/python2.7/dist-packages/salt/utils/data.py", line 504, in filter_by
    val = traverse_dict_and_list(traverse, lookup, [])
  File "/usr/lib/python2.7/dist-packages/salt/utils/data.py", line 578, in traverse_dict_and_list
    for each in key.split(delimiter):
AttributeError: 'list' object has no attribute 'split'

The part in the docs that suggests this should be possible: https://docs.saltstack.com/en/master/ref/modules/all/salt.modules.grains.html#salt.modules.grains.filter_by

grain --

The name of a grain to match with the current system's grains. For example, the value of the "os_family" grain for the current system could be used to pull values from the lookup_dict dictionary.

Changed in version 2016.11.0: The grain value could be a list. The function will return the lookup_dict value for a first found item in the list matching one of the lookup_dict keys.

Trying grain='os_family,osfinger' also does not work.

Same problem at my side, doing grain=‘osfinger,os_family’ does not work either, it will fall back to default then.

The grain value could be a list. The function will return the lookup_dict value for a first found item in the list matching one of the lookup_dict keys.

I cannot find evidence that this is possible because this value is undergone split if it is str or not. A traversal is applied to this string, but this will give you an empty list if you do ‘osfinger:os_family’ because there is no dict behind osfinger that holds the key os_family. And yes, the only delimiter you can apply here is ‘:’.

For completeness, the current version I’m running all that on.

$ salt-call --versions
Salt Version:
           Salt: 3000.1
 
Dependency Versions:
           cffi: 1.12.3
       cherrypy: Not Installed
       dateutil: 2.4.0
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
         Jinja2: 2.10
        libgit2: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.7
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: 2.19
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.17 (default, Nov  7 2019, 10:07:09)
   python-gnupg: 0.4.1
         PyYAML: 3.11
          PyZMQ: 16.0.2
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.2.5
 
System Versions:
           dist: Ubuntu 18.04 bionic
         locale: UTF-8
        machine: x86_64
        release: 4.15.0-96-generic
         system: Linux
        version: Ubuntu 18.04 bionic

@SaltStackSupport Can we please have the stalebot not be so aggressive. 30 days is way too aggressive