salt: [BUG] pam eauth fails with salt-master/salt-api running as user salt on 3006.x

Description Running the salt-master process under the salt user breaks pam eauth as the salt user can’t read /etc/shadow and /etc/gshadow

This issue is documented in SEP19 and Issue 7762

Setup

  • Salt 3006.x installed from Debian packages
  • Same issue will exist with RPM packages but I haven’t specifically tested

Steps to Reproduce the behavior

Configure the salt master to use pam eauth -

$ cat /etc/salt/master.d/ext_auth.conf

external_auth:
  pam:
    testuser:
      - '.*'

$  sudo salt -a pam --username=testuser \* test.ping
password: <password>
Authentication error occurred.

Expected behavior

$ sudo salt -a pam --username=testuser \* test.ping
password: <password>
testhost:
    True

Versions Report

$ salt --versions-report
Salt Version:
          Salt: 3006.1

Python Version:
        Python: 3.10.11 (main, May  5 2023, 02:31:54) [GCC 11.2.0]

Dependency Versions:
          cffi: 1.14.6
      cherrypy: unknown
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.2
       libgit2: Not Installed
  looseversion: 1.0.2
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 22.0
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.9.8
        pygit2: Not Installed
  python-gnupg: 0.4.8
        PyYAML: 5.4.1
         PyZMQ: 23.2.0
        relenv: 0.12.3
         smmap: Not Installed
       timelib: 0.2.4
       Tornado: 4.5.
           ZMQ: 4.3.4

System Versions:
          dist: ubuntu 20.04.6 focal
        locale: utf-8
       machine: x86_64
       release: 5.4.0-148-generic
        system: Linux
       version: Ubuntu 20.04.6 focal

Additional context

This was a known issue with running with the salt-master as a non-root user - documented in SEP19, issue 61770 and has been called out in the eauth documentation for a long time…

There’s a simple fix for Debian derived distributions - adding the salt user to the shadow group allows the salt-master process to read /etc/shadow and resolves the issue. This doesn’t work for RH derived distros.

I’d suggest that for Debian derivatives whether to add salt to the shadow group should be asked through debconf - that way folk that need to use eauth/pam can do so but those who don’t, don’t expose the content of /etc/shadow to the salt user.

I’m not sure how to easily handle this to RPM packages as I’m less familiar with the options there.

Best long term solution would be to have a small separate privileged process for pam eauth that the salt master queries for authentication.

About this issue

  • Original URL
  • State: open
  • Created 4 years ago
  • Reactions: 1
  • Comments: 19 (4 by maintainers)

Most upvoted comments

so, right now this might be more of a documentation issue. if you need pam, change the user back to root. which can be done in /etc/salt/master as blindly allowing access to shadow to a non root user by default is a huge nono, and having the user make a change to the system forces knowledge onto the user that pam needs shadow access.

It was a documentation issue for the 3006.0 release. Now, it’s a breaking change that wasn’t handled or communicated.

I’m also going to disagree that users that need to use pam eauth are better off running the master as root rather than adding the salt user to the shadow group… with the former, you’ve given the salt-master process the ability to do anything as root, with the latter you have only given it the ability to read /etc/shadow and /etc/gshadow.

I think requiring (rather than forcing) knowledge on to the user that pam needs shadow access is better than requiring them to have no understanding and just defaulting to running stuff as root.

Asking through debconf if a user that runs a daemon should be given additional privilege to enable a feature is not unusual and is perfectly reasonable.

I guess for the moment I’ll work on some docs to cover this.

but i like the idea of a separate process for authentication, but that becomes more of a feature request for that part.

Yeah, I think that is probably the way to go longer term as will work across the various Linux distros. I’ll have a think about how to do that.

@major0 - this isn’t about getting group info, this is about being able to authenticate a user. @MartinEmrich your understanding is correct, without doing something such as adding the salt user to the shadow group, it’s only going to be able to auth itself with the pam “login” service.

Whilst not necessarily a “bug” in salt, it’s a “breaking change” in the packaging for 3006.x when the default user that the salt-master process runs as was changed to salt without handling this or documenting it.

I’d also like to actually resolve this issue in a way that isn’t just “run the salt-master as root if you want to use pam eauth” - the most common use case I come across for using pam eauth is when using salt-api - that’s essentially exposing the salt-master on the network… which is exactly the sort of situation where we should be running as an un-privileged user.

@tjyang Yeah, I get that. I’m just saying that your comment is not a fix to this issue. It’s a workaround, but it doesn’t solve the problem, and it’s not related to 3006.9 beyond that version’s change to the default configuration that you mentioned. But, yes, to future people, running as root is the quick workaround, though there are the security implications of doing that of course.

@tjyang The problem revolves around the fact that the master cannot authenticate other users when it itself does not run as root. You’re running the daemon as root. That is why pam eauth works at all, for any Salt version. Change to user: salt and it will not.

Hey guys this is still a problem in 3006.9 right ? I’m unable to login to Salt API on a master running as a non-root user

Found the cause, described here: https://github.com/linux-pam/linux-pam/issues/112#issuecomment-491193418

tl;dr, as I understand it: Non-root-Users can only validate themselves. So if salt ran as user “salt”, only the user “salt” could authenticate via PAM.

What happens? Salt runs a subprocess calling libpam functions. libpam in turn runs its unix_chkpwd binary with that behaviour, which returns an authentication failure when it cannot read the shadow file.

IMHO this is not a bug in Salt, but rather a result of various security design choices in libpam etc.

Why is salt trying to read the group file at all? Shouldn’t it being going through the nss interface in the same way that getent does?

getent group salt

PAM+NSS is nice because it can transparently handle NIS/NIS+, LDAP, ActiveDirectory, etc such that all standard libc calls still work. Tools such as getent, id, passwd transparently work for any of these services w/out any special configurations. Why is salt doing something non-standard here?

thank goodness I am using debian, otherwise the last 4 hours of my life would have been for naught!