fluent-bit: Systemd input doesn't work properly with SELinux policy deny_execmem

Bug Report

When first enabling the deny_execmem SELinux policy (a policy which will deny mapped memory to be both executable and writable at the same time). And after that running fluent-bit with the systemd input, it will get denied by this policy and just hang there forever doing nothing. I ran into this when upgrading from fluent-bit 1.9.6 to 2.0.4 (and I confirmed this is still the case with 2.0.6)

To Reproduce

[root@localhost ~]# /opt/fluent-bit/bin/fluent-bit -i systemd -o stdout > /dev/null 
Fluent Bit v2.0.6
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2022/12/02 09:55:20] [ info] [fluent bit] version=2.0.6, commit=, pid=36650
[2022/12/02 09:55:20] [ info] [storage] ver=1.3.0, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2022/12/02 09:55:20] [ info] [cmetrics] version=0.5.7
[2022/12/02 09:55:20] [ info] [ctraces ] version=0.2.5
[2022/12/02 09:55:20] [ info] [input:systemd:systemd.0] initializing
[2022/12/02 09:55:20] [ info] [input:systemd:systemd.0] storage_strategy='memory' (memory only)
[2022/12/02 09:55:20] [ info] [sp] stream processor started
[2022/12/02 09:55:20] [ info] [output:stdout:stdout.0] worker #0 started
^C[2022/12/02 09:55:21] [engine] caught signal (SIGINT)
[2022/12/02 09:55:21] [ warn] [engine] service will shutdown in max 5 seconds
[2022/12/02 09:55:21] [ info] [input] pausing systemd.0
[2022/12/02 09:55:22] [ info] [engine] service has stopped (0 pending tasks)
[2022/12/02 09:55:22] [ info] [input] pausing systemd.0
[2022/12/02 09:55:22] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2022/12/02 09:55:22] [ info] [output:stdout:stdout.0] thread worker #0 stopped
[root@localhost ~]# setsebool -P deny_execmem on
[root@localhost ~]# /opt/fluent-bit/bin/fluent-bit -i systemd -o stdout
Fluent Bit v2.0.6
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

It will hang like this forever and it doesn’t respond to SIGTERM either, has to be killed from another session. I also added some additional logs from /var/log/audit/audit.log

[root@localhost log]# cat /var/log/audit/audit.log | grep -i fluent
type=AVC msg=audit(1669972062.414:96): avc:  denied  { execmem } for  pid=1817 comm="fluent-bit" scontext=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 tclass=process permissive=0
type=SYSCALL msg=audit(1669972062.414:96): arch=c000003e syscall=10 success=no exit=-13 a0=7f5ec3aab000 a1=800000 a2=7 a3=ffffffffffffffc0 items=0 ppid=1692 pid=1817 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=1 comm="fluent-bit" exe="/opt/fluent-bit/bin/fluent-bit" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null)ARCH=x86_64 SYSCALL=mprotect AUID="root" UID="root" GID="root" EUID="root" SUID="root" FSUID="root" EGID="root" SGID="root" FSGID="root"

Steps to reproduce the problem:

• Install a SELinux distro, I initially ran into it on Fedora CoreOS with the docker images. But I confirmed this behavior on CentOS 9 as well.
• Install fluent-bit, I followed the instructions on the website https://docs.fluentbit.io/manual/installation/linux/redhat-centos#install-on-redhat-centos
• Enable the deny_execmem flag using setsebool -P deny_execmem on
• Run fluent-bit with the systemd input enabled /opt/fluent-bit/bin/fluent-bit -i systemd -o stdout for example

Expected behavior I would expect fluent-bit to run as normal

Your Environment

• Version used: 2.0.6
• Configuration: None, I narrowed it down enough that it can run from command line right away fluent-bit -i systemd -o stdout
• Environment name and version (e.g. Kubernetes? What version?):
• Server type and version:
• Operating System and version: CentOS 9
• Filters and plugins: systemd

Additional context I ran into this while upgrading fluent-bit on a hardened server, where we set a lot of extra SELinux policies. And it basically stopped working completely with our existing configuration.