falco: Facing error in Falco helm installation on AWS EKS

Facing same issue for EKS 1.22 , Faclo 0.32.2 Attached logs for reference

* Setting up /usr/src links from host
* Running falco-driver-loader for: falco version=0.32.2, driver version=2.0.0+driver
* Running falco-driver-loader with: driver=module, compile=yes, download=yes

================ Cleaning phase ================

* 1. Check if kernel module 'falco' is still loaded:
- OK! There is no 'falco' module loaded.

* 2. Check all versions of kernel module 'falco' in dkms:
- There are some versions of 'falco' module in dkms.

* 3. Removing all the following versions from dkms:
2.0.0+driver

- Removing 2.0.0+driver...

------------------------------
Deleting module version: 2.0.0+driver
completely from the DKMS tree.
------------------------------
Done.

- OK! Removing '2.0.0+driver' succeeded.


[SUCCESS] Cleaning phase correctly terminated.

================ Cleaning phase ================

* Looking for a falco module locally (kernel 5.4.204-113.362.amzn2.aarch64)
* Trying to download a prebuilt falco module from https://download.falco.org/driver/2.0.0%2Bdriver/aarch64/falco_amazonlinux2_5.4.204-113.362.amzn2.aarch64_1.ko
curl: (22) The requested URL returned error: 404 
Unable to find a prebuilt falco module

Please provide steps to resolve this.

_Originally posted by @ap-mx-git in https://github.com/falcosecurity/falco/issues/1803#issuecomment-1219522445_

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 2
  • Comments: 22 (9 by maintainers)

Most upvoted comments

Same issue, it broke yesterday when I upgraded my kube cluster to 1.22 and nodes were re-created to the new kernel: 5.4.219-126.411.amzn2.x86_64 This used to work with my previous nodes kernel 5.4.209-116.367.amzn2.x86_64

I know we need to wait for the kernel to be listed here for falco-driver-loader initConatiner to work: https://download.falco.org/?prefix=driver/2.0.0%2Bdriver/x86_64/

I also know I could try to recreate my cluster and force my nodes to use a specific AMI (or even installing some headers manually on the nodes) but I really dont want to go down that path.

Is there a way to tell falco (via helm install charts) to use a specific driver maybe? would that work with close enough kernel versions like the ones I have listed above?

Hi team, is there any update on this? Just installed a fresh chart 2.3.0 - published 14th Nov (with Falco 0.33) and facing the same issue as above…

  • EKS 1.22
  • AMI used: amazon-eks-node-1.22-v20221104

Thanks

This is also erroring out for EKS with version Kubernetes version v1.23.9-eks-ba74326 and Falco 0.33.0 or 0.32.2 and 0.31.1. Here is what I see on container :

-------------------------------------------------------------------------
* Setting up /usr/src links from host
* Running falco-driver-loader for: falco version=0.33.0, driver version=3.0.1+driver, arch=x86_64, kernel release=5.4.217-126.408.amzn2.x86_64, kernel version=1
* Running falco-driver-loader with: driver=module, compile=yes, download=yes

================ Cleaning phase ================

* 1. Check if kernel module 'falco' is still loaded:
- OK! There is no 'falco' module loaded.

* 2. Check all versions of kernel module 'falco' in dkms:
- There are some versions of 'falco' module in dkms.

* 3. Removing all the following versions from dkms:
3.0.1+driver

- Removing 3.0.1+driver...

------------------------------
Deleting module version: 3.0.1+driver
completely from the DKMS tree.
------------------------------
Done.

- OK! Removing '3.0.1+driver' succeeded.


[SUCCESS] Cleaning phase correctly terminated.

================ Cleaning phase ================

* Looking for a falco module locally (kernel 5.4.217-126.408.amzn2.x86_64)
* Filename 'falco_amazonlinux2_5.4.217-126.408.amzn2.x86_64_1.ko' is composed of:
 - driver name: falco
 - target identifier: amazonlinux2
 - kernel release: 5.4.217-126.408.amzn2.x86_64
 - kernel version: 1
* Trying to download a prebuilt falco module from https://download.falco.org/driver/3.0.1%2Bdriver/x86_64/falco_amazonlinux2_5.4.217-126.408.amzn2.x86_64_1.ko
curl: (22) The requested URL returned error: 404 
Unable to find a prebuilt falco module
* Trying to dkms install falco module with GCC /usr/bin/gcc
DIRECTIVE: MAKE="'/tmp/falco-dkms-make'"

Creating symlink /var/lib/dkms/falco/3.0.1+driver/source ->
                 /usr/src/falco-3.0.1+driver

DKMS: add completed.
* Running dkms build failed, couldn't find /var/lib/dkms/falco/3.0.1+driver/build/make.log (with GCC /usr/bin/gcc)
* Trying to dkms install falco module with GCC /usr/bin/gcc-8
DIRECTIVE: MAKE="'/tmp/falco-dkms-make'"
* Running dkms build failed, couldn't find /var/lib/dkms/falco/3.0.1+driver/build/make.log (with GCC /usr/bin/gcc-8)
* Trying to dkms install falco module with GCC /usr/bin/gcc-6
DIRECTIVE: MAKE="'/tmp/falco-dkms-make'"
* Running dkms build failed, couldn't find /var/lib/dkms/falco/3.0.1+driver/build/make.log (with GCC /usr/bin/gcc-6)
* Trying to dkms install falco module with GCC /usr/bin/gcc-5
DIRECTIVE: MAKE="'/tmp/falco-dkms-make'"
* Running dkms build failed, couldn't find /var/lib/dkms/falco/3.0.1+driver/build/make.log (with GCC /usr/bin/gcc-5)
* Trying to load a system falco module, if present
Consider compiling your own falco driver and loading it or getting in touch with the Falco community
Error from server (BadRequest): container "falco" in pod "falco-lcvg4" is waiting to start: PodInitializing

The original issue was related to missing pre-built kernel modules. It isn’t relevant anymore I think.

We are facing the same issue with the kernel 5.4.209-116.367.amzn2.x86_64. We tried the suggested command to install kernel module yum install kernel-devel.

After that falco runs but its not able to capture any event from the pod as suggested by amazon in this link

https://aws.amazon.com/blogs/containers/implementing-runtime-security-in-amazon-eks-using-cncf-falco/

which are in a demo nginx pod I ran the following

touch /etc/2 cat /etc/shadow > /dev/null 2>&1

EKS 1.22 falco 0.32.2.

Kindly suggest the remedies as this is urgent for us.