salt: `saltutil.sync_all`: ERROR: Path 'top.sls' is not absolute

Description of Issue/Question

As modules keep disappearing as discussed in https://github.com/saltstack/salt/issues/35443, when it happens, there’s a runner executing saltutil.sync_all.

However, today it failed and I did it manually. Having the same configuration as always, I got the errors below:

$ sudo salt 'edge*' saltutil.sync_all
edge01.yvr01:
    ERROR: Path 'top.sls' is not absolute
edge01.bjm01:
    ERROR: Path 'top.sls' is not absolute
edge01.per01:
    ERROR: Path 'top.sls' is not absolute
edge01.flw01:
    ERROR: Path 'top.sls' is not absolute

As this was a production environment, I had to remediate the problem quickly and restart the proxy processes. After the restart, I run again saltutil.sync_all and the error did not appear anymore, everything came back to normal.

The top.sls file is under /etc/salt/pillar/top.sls. In the master:

pillar_roots:
  base:
    - /etc/salt/pillar
    - /etc/salt/proxy-master/pillar

The log level on the master was set to warning and I did not see anything related. Same level on the proxy, all I was able to see:

2017-01-27 09:51:59,394 [salt.minion      ][ERROR   ][29299] A command in 'saltutil.sync_all' had a problem: Path 'top.sls' is not absolute
2017-01-27 09:52:00,090 [salt.minion      ][ERROR   ][29483] A command in 'saltutil.sync_all' had a problem: Path 'top.sls' is not absolute

I’m wondering how this and https://github.com/saltstack/salt/issues/35443 are related and why after the restart did not complain anymore about Path 'top.sls' is not absolute?

Salt Version

$ sudo salt --versions-report
Salt Version:
           Salt: 2016.11.0

Dependency Versions:
           cffi: 0.8.6
       cherrypy: 10.0.0
       dateutil: 2.2
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.2
   mysql-python: Not Installed
      pycparser: 2.17
       pycrypto: 2.6.1
         pygit2: Not Installed
         Python: 2.7.9 (default, Mar  1 2015, 12:57:24)
   python-gnupg: Not Installed
         PyYAML: 3.12
          PyZMQ: 14.4.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5

System Versions:
           dist: debian 8.4
        machine: x86_64
        release: 4.4.17-cloudflare
         system: Linux
        version: debian 8.4

Thanks, Mircea

About this issue

Original URL
State: closed
Created 7 years ago
Comments: 16 (15 by maintainers)

Most upvoted comments

Sounds good. Shall do!

mirceaulinic on Mar 14, 2018

@mirceaulinic my guess would be that this is a bug that has since been fixed. As @gtmanfred noted, __gen_opts() is where we set the effective state_top for the state run.

terminalmage on Mar 13, 2018

You’re asking the fileclient to return top.sls, not salt://top.sls. When there is no protocol, the fileclient doesn’t know whether it needs to make a request to the salt fileserver, a webserver, or whatever, so it assumes the path is a local file on the minion. Only absolute paths are supported when “caching” local files (the minion doesn’t actually cache them, just returns back the specified path if it exists), hence the error.

terminalmage on Mar 3, 2018

Thanks for the analysis @mirceaulinic

@saltstack/team-core does anyone have any idea what is going on here? It looks like this may have also been hit at some point in the past here #17286

Thanks, Daniel

gtmanfred on Aug 1, 2017

Yup, you are correct somewhere it is not expanding that top.sls to be the full path of /etc/salt/pillar/top.sls, we will just need to track it down. If you run across it in your testing environment, with the log level set to debug it should give a stacktrace which would show us the path that the config traveled thru and was not expanded.

Thanks for you help! Daniel

gtmanfred on Jan 27, 2017