salt: [BUG] pkgrepo.managed fails to configure ppa on Ubuntu 20.10

Still happening with 3004.1 on 22.04 (jammy). I’m not sure whether this is the right fix, but this is the patch I’m using locally, which seems to work and also doesn’t change the PPA repo entry every time it runs:

--- /usr/lib/python3/dist-packages/salt/modules/aptpkg.py.orig  2022-05-04 18:45:18.181727709 -0700
+++ /usr/lib/python3/dist-packages/salt/modules/aptpkg.py       2022-05-04 19:31:25.042603741 -0700
@@ -1847,9 +1847,11 @@
             if HAS_SOFTWAREPROPERTIES:
                 try:
                     if hasattr(softwareproperties.ppa, "PPAShortcutHandler"):
-                        repo = softwareproperties.ppa.PPAShortcutHandler(repo).expand(
-                            dist
-                        )[0]
+                        handler = softwareproperties.ppa.PPAShortcutHandler(repo)
+                        if hasattr(handler, "expand"):
+                            repo = handler.expand(dist)[0]
+                        else:
+                            repo = handler.SourceEntry().line
                     else:
                         repo = softwareproperties.ppa.expand_ppa_line(repo, dist)[0]
                 except NameError as name_error:
@@ -1930,7 +1932,11 @@
                 repo = LP_SRC_FORMAT.format(owner_name, ppa_name, dist)
         else:
             if hasattr(softwareproperties.ppa, "PPAShortcutHandler"):
-                repo = softwareproperties.ppa.PPAShortcutHandler(repo).expand(dist)[0]
+                handler = softwareproperties.ppa.PPAShortcutHandler(repo)
+                if hasattr(handler, "expand"):
+                    repo = handler.expand(dist)[0]
+                else:
+                    repo = handler.SourceEntry().line
             else:
                 repo = softwareproperties.ppa.expand_ppa_line(repo, dist)[0]

@@ -2657,9 +2663,11 @@
         else:
             if HAS_SOFTWAREPROPERTIES:
                 if hasattr(softwareproperties.ppa, "PPAShortcutHandler"):
-                    repo = softwareproperties.ppa.PPAShortcutHandler(repo).expand(dist)[
-                        0
-                    ]
+                    handler = softwareproperties.ppa.PPAShortcutHandler(repo)
+                    if hasattr(handler, "expand"):
+                        repo = handler.expand(dist)[0]
+                    else:
+                        repo = handler.SourceEntry().line
                 else:
                     repo = softwareproperties.ppa.expand_ppa_line(repo, dist)[0]
             else:

The bug can be bypassed by switching to a regular type of repository instead of ppa. E.g.:

formula.python.ubuntu.deadsnakes.repo:
  pkgrepo.managed:
    - ppa: deadsnakes/ppa
    - refresh: true

to

formula.python.ubuntu.deadsnakes.repo:
  pkgrepo.managed:
    - name: "deb [signed-by=/etc/apt/keyrings/deadsnakes.gpg] https://ppa.launchpadcontent.net/deadsnakes/ppa/ubuntu {{ grains.oscodename }} main"
    - file: /etc/apt/sources.list.d/deadsnakes.list
    - key_url: https://keyserver.ubuntu.com/pks/lookup?op=get&search=0xba6932366a755776
    - aptkey: False
    - refresh: true

Confirmed this exists in a brand new Ubuntu 22.04.1 LTS (installed on OSX using multipass). FWIW salt was installed using the bash script (not apt).

$ salt-minion --version
salt-minion 3005.1

$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.1 LTS"

Note I’ve done apt-get update && apt-get upgrade && service salt-minion restart but that didn’t help.

The patch in #59065 (comment) fixed it for me and I applied it using a salt state (on my masterless-minion).

# Patches SaltStack Issue 59065
# See https://github.com/saltstack/salt/issues/59065#issuecomment-1366575542

{% if grains.saltversion == "3005.1" %}

# Just in case there is no patch command installed
patch:
  pkg.installed

# We use directory name here (saltpath) to apply patches that touch multiple files
patch_applied_59065:
  file.patch:
    - name: '{{ grains.saltpath }}'
    - source: salt://patches/files/saltstack_issue_59065.diff
    - strip: 6
    - require:
        - pkg: patch

# Restart salt minion after the patch is applied and the config option is set
restart_salt_minion:
  cmd.run:
    - name: 'salt-call service.restart salt-minion'
    - bg: true
    - onchanges:
      - patch_applied_59065
{%- endif %}

Where patches/files/saltstack_issue_59065.diff is the patch as referenced above.

This begs the question (for another thread), should the salt-minion issue an “unsupported os version for salt-minion” warning when running jobs to help incompetent people, like me, to do a proper upgrade? 😃

Confirmed still happening in 3005.1

# salt-minion --version
salt-minion 3005.1

# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.1 LTS"

Ummm… I have to amend the above. I have two ubuntu systems, both started their lives as 20.04LTS and have gone through do-release-upgrade. Both have 3005.1 installed.

However, I got the error on one of the systems but not on the other.

Here’s the diff between them:

Server WITHOUT error:

# cat /etc/apt/sources.list.d/salt.list
deb [signed-by=/etc/apt/keyrings/salt-archive-keyring.gpg arch=amd64] https://repo.saltproject.io/salt/py3/ubuntu/22.04/amd64/latest jammy main

As you can see, on the server that doesn’t fail, this is a onedir package.

Server WITH error:

# cat /etc/apt/sources.list.d/salt.list
deb [signed-by=/usr/share/keyrings/salt-archive-keyring.gpg] https://repo.saltproject.io/py3/ubuntu/20.04/amd64/latest focal main

As you can see, this is a legacy package for Ubuntu 20 (focal)… not 22 (jammy)

So… I updated my apt-source by following the install guide (https://docs.saltproject.io/salt/install-guide/en/latest/topics/install-by-operating-system/ubuntu.html#install-onedir-packages-of-salt-on-ubuntu-22-04-jammy) to update to a onedir package.

After running the following…

apt-get update 
apt-get dist-upgrade 
systemctl restart salt-minion

… no more error.

Moral of the story… Don’t forget to update third-party apt sources when doing a release upgrade. 😕

Confirmed that this is still a problem on Ubuntu 22.04 + Salt 3004.1. The patch in https://github.com/saltstack/salt/issues/59065#issuecomment-1118112165 fixes it. It would be great if this could be committed.

Seeing it in Ubuntu 22.04 pre-release, salt-minion 3004 (installed from https://repo.saltproject.io/py3/ubuntu/20.04/amd64/latest focal InRelease:

chromium ppa:
  pkgrepo.managed:
    - ppa: phd/chromium-browser

/srv/salt$ sudo salt-call --local state.apply work_laptop.programs.chromium
[ERROR   ] An exception occurred in this state: Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/state.py", line 2179, in call
    ret = self.states[cdata["full"]](
  File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 149, in __call__
    return self.loader.run(run_func, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1201, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1216, in _run_as
    return _func_or_method(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1249, in wrapper
    return f(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/states/pkgrepo.py", line 400, in managed
    pre = __salt__["pkg.get_repo"](repo=repo, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 149, in __call__
    return self.loader.run(run_func, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1201, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1216, in _run_as
    return _func_or_method(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/modules/aptpkg.py", line 1850, in get_repo
    repo = softwareproperties.ppa.PPAShortcutHandler(repo).expand(
AttributeError: 'PPAShortcutHandler' object has no attribute 'expand'

local:
----------
          ID: chromium ppa
    Function: pkgrepo.managed
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/lib/python3/dist-packages/salt/state.py", line 2179, in call
                  ret = self.states[cdata["full"]](
                File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 149, in __call__
                  return self.loader.run(run_func, *args, **kwargs)
                File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1201, in run
                  return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
                File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1216, in _run_as
                  return _func_or_method(*args, **kwargs)
                File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1249, in wrapper
                  return f(*args, **kwargs)
                File "/usr/lib/python3/dist-packages/salt/states/pkgrepo.py", line 400, in managed
                  pre = __salt__["pkg.get_repo"](repo=repo, **kwargs)
                File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 149, in __call__
                  return self.loader.run(run_func, *args, **kwargs)
                File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1201, in run
                  return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
                File "/usr/lib/python3/dist-packages/salt/loader/lazy.py", line 1216, in _run_as
                  return _func_or_method(*args, **kwargs)
                File "/usr/lib/python3/dist-packages/salt/modules/aptpkg.py", line 1850, in get_repo
                  repo = softwareproperties.ppa.PPAShortcutHandler(repo).expand(
              AttributeError: 'PPAShortcutHandler' object has no attribute 'expand'
     Started: 15:45:21.422122
    Duration: 2663.127 ms
     Changes:   

Summary for local
------------
Succeeded: 0
Failed:    1
------------
Total states run:     1
Total run time:   2.663 s

FYI for anyone finding this issue, looks like the problem was merely punted forward into Ubuntu 21.04 which includes Salt 3002.2. So the problem isn’t solved but more like postponed. Installing 3003.3 does not resolve the issue, but since the distribution includes 3002.2 that’s what I’ll list below.

%> apt show salt-minion                
Package: salt-minion
Version: 3002.2+dfsg1-1
Priority: optional
Section: universe/admin
Source: salt
Origin: Ubuntu
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Debian Salt Team <pkg-salt-team@alioth-lists.debian.net>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 119 kB
Pre-Depends: init-system-helpers (>= 1.54~)
Depends: dctrl-tools, lsb-base (>= 3.0-6), python3-pycryptodome (>= 2.6) | python3-crypto (>= 2.6), python3-systemd | sysvinit-core, python3-zmq (>= 13.1.0), salt-common (= 3002.2+dfsg1-1), python3:any
Recommends: debconf-utils, dmidecode, e2fsprogs, sfdisk
Suggests: python3-augeas
Homepage: http://saltstack.org/
Download-Size: 27.7 kB
APT-Manual-Installed: yes
APT-Sources: http://fi.archive.ubuntu.com/ubuntu hirsute/universe amd64 Packages
Description: client package for salt, the distributed remote execution system

%> apt show python3-software-properties        
Package: python3-software-properties
Version: 0.99.10
Priority: optional
Section: python
Source: software-properties
Origin: Ubuntu
Maintainer: Michael Vogt <michael.vogt@ubuntu.com>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 197 kB
Depends: gpg, iso-codes, lsb-release, python3, python3-apt (>= 0.6.20ubuntu16), python3-gi, python3-launchpadlib, python3:any
Recommends: unattended-upgrades
Task: server, ubuntu-desktop-minimal, ubuntu-desktop, cloud-image, server-raspi, kubuntu-desktop, xubuntu-core, xubuntu-desktop, lubuntu-desktop, ubuntustudio-desktop-core, ubuntustudio-desktop, ubuntukylin-desktop, ubuntu-mate-core, ubuntu-mate-desktop, ubuntu-budgie-desktop, ubuntu-budgie-desktop-raspi
Download-Size: 32.0 kB
APT-Manual-Installed: yes
APT-Sources: http://fi.archive.ubuntu.com/ubuntu hirsute/main amd64 Packages
Description: manage the repositories that you install software from