falco: Falco won't start on 0.36 - /sys/devices/system/cpu/cpu8/online: No such file or directory

Wohoo, it’s running… forgot to override the driver installer image

    driver:
      kind: ebpf
      loader:
        initContainer:
          tag: 0.36.1-rc1
    image:
      tag: 0.36.1-rc1

During the community call an hour ago we proposed a patch release 0.36.1 for Falco that will surely cover this issue 😃 expect it in a couple of weeks!

no its not working

Awesome, I’ll remove the image override from the Flux HelmRelease when I get home and report back if there’s any issues.

Thank you very much for testing it!

@Andreagit97 thanks for the suggestion but I am experiencing this issue on an older kernel version (4.x).

I am so happy about this! So, I’ll come back and close this issue as solved once the libs PR is merged and libs are bumped on Falco master 😃 At least, then you’ll have Falco development images with working bpf engine!

I think this is good, I just bind mounted the file in one of the nodes🤣

Sat Sep 30 20:30:35 2023: Falco version: 0.37.0-48+31d6232 (x86_64)
Sat Sep 30 20:30:35 2023: Falco initialized with configuration file: /etc/falco/falco.yaml
Sat Sep 30 20:30:35 2023: Loading rules from file /etc/falco/falco_rules.yaml
Sat Sep 30 20:30:35 2023: Loading rules from file /etc/falco/falco-incubating_rules.yaml
Sat Sep 30 20:30:36 2023: Loading rules from file /etc/falco/falco-sandbox_rules.yaml
Sat Sep 30 20:30:36 2023: Loading rules from file /etc/falco/rules.d/rules-custom.yaml
Sat Sep 30 20:30:36 2023: The chosen syscall buffer dimension is: 8388608 bytes (8 MBs)
Sat Sep 30 20:30:36 2023: gRPC server threadiness equals to 8
Sat Sep 30 20:30:36 2023: Starting health webserver with threadiness 8, listening on port 8765
Sat Sep 30 20:30:36 2023: Loaded event sources: syscall
Sat Sep 30 20:30:36 2023: Enabled event sources: syscall
Sat Sep 30 20:30:36 2023: Opening 'syscall' source with BPF probe. BPF probe path: /root/.falco/falco-bpf.o
Sat Sep 30 20:30:36 2023: Starting gRPC server at unix:///run/falco/falco.sock

No container? 😥

Unfortunately not; for that, we would need to merge the libs PR and then merge the libs bump PR in Falco 😦 But i would love to test the fix before merging it in libs, and there we need your help ahah Thank you very much!

No container? 😥 I’ll try to work around it later.

It’s the same - cat /sys/devices/system/cpu/possible is 0-127 🤯

The patch is ready: https://github.com/falcosecurity/libs/pull/1373 Still, i’d love to understand what’s going on here 😄

🤯

At least if it didn’t work in the previous version I’d suspect something else, kernel upgrade for example. It doesn’t help that this seems to work in previous version and not this one.

Let me know what else I could try.

I’m just rolling back to the previous version for now

Here it is

getconf _NPROCESSORS_ONLN
8
getconf _NPROCESSORS_CONF
8

It is not starting on any node, and they’ve not had any core count changed lately. I’ve even rebooted one node to make sure it’s not reboot related.

It’s a virtual machine with CPU hotplug enabled, could that have something to do with the possible value being at 127?

I think so; it seems like a way for the vm to allow increasing number of online cpus (ie: CPUs made available to the vm) without the need to reboot. I think Falco is not able to correctly manage this situation at the moment. Fact is, i don’t get how could Falco 0.35.1 work in the very same situation.

I don’t understand, so the real issue is:

2023-09-26T20:36:20+01:00 libpman: ring buffer map type is not supported (errno: 22 | message: Invalid argument) ?

Yes, when using modern eBPF

When using old eBPF, the error is

Error: can't open /sys/devices/system/cpu/cpu8/online: No such file or directory

Thanks for picking it up @FedeDP I forgot to mention - eBPF