security-profiles-operator: AppArmor profiles not loading on worker nodes in K8S cluster

What happened:

We are using using AppArmor in our project. In K8S cluster, After installing the SPO and enabing the AppArmor feature, i am creating the AppArmor profiles ,profiles are created successfully however those profiles are not getting loaded in the worker nodes.

What you expected to happen:

Once i create the AppArmor profiles, the profiles should get loaded in all the worker nodes so that whenever i launch any application pod on worker nodes , application should get validated by the profiles before doing any system operations.

How to reproduce it (as minimally and precisely as possible):

Configuration: VMs, 1 master and 1 worker

Enbled AppArmor on both the VM’s

kmaster@kmaster:~$ kubectl get nodes -o=jsonpath='{range .items[*]}{@.metadata.name}: {.status.conditions[?(@.reason=="KubeletReady")].message}{"\n"}{end}'
kmaster: kubelet is posting ready status. AppArmor enabled
kworker1: kubelet is posting ready status. AppArmor enabled

Installed SPO: SPO : v10.4.3

$ kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.9.1/cert-manager.yaml
$ kubectl --namespace cert-manager wait --for condition=ready pod -l app.kubernetes.io/instance=cert-manager
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/security-profiles-operator/main/deploy/operator.yaml

Enabled AppArmor: kmaster@kmaster:~/testProfiles/installFiles/UsingSPO$ cat config.yaml

---
apiVersion: security-profiles-operator.x-k8s.io/v1alpha1
kind: SecurityProfilesOperatorDaemon
metadata:
  name: spod
  namespace: security-profiles-operator
spec:
  selinuxTypeTag: ""
  enableSelinux: false
  enableLogEnricher: false
  verbosity: 1
  enableAppArmor: true

Profile creation:

---
apiVersion: security-profiles-operator.x-k8s.io/v1alpha1
kind: AppArmorProfile
metadata:
  name: apparmorprofile-test100
  annotations:
    description: Block writing to any files in the disk.
spec:
  policy: |
    #include <tunables/global>

    profile test-profile flags=(attach_disconnected) {
      #include <abstractions/base>

      file,

      # Deny all file writes.
      deny /** w,
    }

Once i apply the profile, follwoign errors are coming in the SPOD and Security profile Operator pod logs: spod log message:

I0928 20:36:35.539155   45582 apparmorprofile.go:112] apparmor-spod "msg"="Reconciling AppArmorProfile" "apparmorprofile"="apparmorprofile-test100" "namespace"="security-profiles-operator"
E0928 20:36:35.539312   45582 apparmorprofile.go:180] apparmor-spod "msg"="cannot load profile into node" "error"="open /etc/apparmor.d/spo-apparmorprofile-test100: no such file or directory" "apparmorprofile"="apparmorprofile-test100" "namespace"="security-profiles-operator"
E0928 20:36:35.539351   45582 controller.go:326]  "msg"="Reconciler error" "error"="cannot load profile into node: open /etc/apparmor.d/spo-apparmorprofile-test100: no such file or directory" "appArmorProfile"={"name":"apparmorprofile-test100","namespace":"security-profiles-operator"} "controller"="apparmorprofile" "controllerGroup"="security-profiles-operator.x-k8s.io" "controllerKind"="AppArmorProfile" "name"="apparmorprofile-test100" "namespace"="security-profiles-operator" "reconcileID"="5a0bd06a-4274-4b15-8f47-fe92b09f354d"

security-profiles-operator pod message:

E0928 20:40:06.667191       1 controller.go:326]  "msg"="Reconciler error" "error"="getting owner profile: the node status owner is of an unknown kind" "controller"="nodestatus" "controllerGroup"="security-profiles-operator.x-k8s.io" "controllerKind"="SecurityProfileNodeStatus" "name"="apparmorprofile-test100-kworker1" "namespace"="security-profiles-operator" "reconcileID"="b316d16e-52a0-4857-870f-a0fc78c079c1" "securityProfileNodeStatus"={"name":"apparmorprofile-test100-kworker1","namespace":"security-profiles-operator"}
I0928 20:40:13.569728       1 ca.go:62] spod-config "msg"="Using cert-manager as certificate provider"

Anything else we need to know?:

Also, i wanted to know how to unload a AppArmor profiles , once the loading is successful.

Environment:

  • Cloud provider or hardware configuration: Using VBOX VM’s on my local laptop

  • OS (e.g: cat /etc/os-release): kmaster@kmaster:~/testProfiles/installFiles/UsingSPO$ cat /etc/os-release

PRETTY_NAME="Ubuntu 22.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.1 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
  • Kernel (e.g. uname -a): kmaster@kmaster:~/testProfiles/installFiles/UsingSPO$ uname -a Linux kmaster 5.15.0-48-generic #54-Ubuntu SMP Fri Aug 26 13:26:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

  • Others:

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 18 (7 by maintainers)

Most upvoted comments

@juggarnautss we have released a version Today which means the official operator image is out with the necessary fixes for AppArmor to work.

I will try to invest some time on the settings later on this week. That isn’t a fix as such, but rather a sensible default that would not give users the impression of the operator not working.

As for the chart, it is based in dir /deploy/helm of this repo. At the moment, we don’t expose an enableAppArmor attribute. So feel free to propose a PR for it, otherwise I will try to get around to it at some point this week.

SPO is currently stable with AppArmor enabled. Also the profiles are getting loaded and unloaded properly across worker nodes. Hence, marking this issue as closed.

@pjbgf

Hi Paulo,

I have raised the feature required to enable apparmor by default. https://github.com/kubernetes-sigs/security-profiles-operator/issues/1320

Thanks and regards, Sirin