fluent-bit: Oracle Linux on OKE cluster, : Unsupported system page size

Bug Report

First , want to reference #4270.

I am using ARM64 Oracle Linux 7.9 on an Oracle Cloud Kubernetes cluster.

I tried installing the Sumo Logic’s Helm chart (https://github.com/SumoLogic/sumologic-kubernetes-collection/blob/main/deploy/helm/sumologic/Chart.yaml), but received the below error message.

Upon that failed install, I saw #4270, which clued me in that there was a fix. I looked at the Sumo Logic Helm chart to reference what release of fluent-bit was being used, and saw that it was an older release - 0.20.2 (though, from what I saw, still a release that should include the fixes from #4270). Upon that discovery, I attempted installing the latest release of the fluent-bit chart directly. I received the same error.

Brief snippet of error message:

<jemalloc>: Unsupported system page size
Error in GnuTLS initialization: ASN1 parser: Element was not found.
<jemalloc>: Unsupported system page size
<jemalloc>: Unsupported system page size
<jemalloc>: Unsupported system page size
malloc: Cannot allocate memory

Pod logs / error message is attached: fluentbit-fluent-bit-5z9r4.log

Node output:

[root@oke-cs56bxasraq-ntlw7x5fjfq-sbmqgfzjhbq-0 /]# getconf PAGESIZE
65536

Environment details:

  • Version of fluent-bit used:
  • Sumo Logic uses: 0.20.2
  • I also tried directly installed fluent-bit Helm Chart version: 0.20.4

  • Kubernetes version: v1.23.4

  • Node:

[root@oke-cs56bxasraq-ntlw7x5fjfq-sbmqgfzjhbq-0 /]# uname -a
Linux oke-cs56bxasraq-ntlw7x5fjfq-sbmqgfzjhbq-0 5.4.17-2136.307.3.1.el7uek.aarch64 #2 SMP Mon May 9 17:34:42 PDT 2022 aarch64 aarch64 aarch64 GNU/Linux
[root@oke-cs56bxasraq-ntlw7x5fjfq-sbmqgfzjhbq-0 /]# cat /etc/*elease
Oracle Linux Server release 7.9
NAME="Oracle Linux Server"
VERSION="7.9"
ID="ol"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.9"
PRETTY_NAME="Oracle Linux Server 7.9"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:oracle:linux:7:9:server"
HOME_URL="https://linux.oracle.com/"
BUG_REPORT_URL="https://bugzilla.oracle.com/"

ORACLE_BUGZILLA_PRODUCT="Oracle Linux 7"
ORACLE_BUGZILLA_PRODUCT_VERSION=7.9
ORACLE_SUPPORT_PRODUCT="Oracle Linux"
ORACLE_SUPPORT_PRODUCT_VERSION=7.9
Red Hat Enterprise Linux Server release 7.9 (Maipo)
Oracle Linux Server release 7.9

To Reproduce

  • Have K8s cluster using Oracle Linux image for nodes
  • Try to install Sumo Logic helm chart or latest release of fluent-bit directly

Thanks for any help you can provide!

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 20 (11 by maintainers)

Most upvoted comments

@patrick-stephens hi, thx for your reply, I reproduced it again:

fluentbit 2.0.9

  • docker metadata:
[root@host-x-x-x-x ~]# docker pull fluent/fluent-bit:2.0.9
2.0.9: Pulling from fluent/fluent-bit
5f80a38cb015: Pull complete
a839fcbfad1b: Pull complete
4d704a88b82e: Pull complete
e40cb4c669fc: Pull complete
0d2a2f1491cc: Pull complete
71b31c8271be: Pull complete
Digest: sha256:d370ad358bc8c392d157ed89eaf4555d49041bf86b29a6a8150981d04d286be8
Status: Downloaded newer image for fluent/fluent-bit:2.0.9
docker.io/fluent/fluent-bit:2.0.9
[root@host-x-x-x-x ~]# docker inspect docker.io/fluent/fluent-bit:2.0.9
[
    {
        "Id": "sha256:f35881e560a62e83b174d49dad6aa8e80deb547ea7938132358beae8714ab4f4",
        "RepoTags": [
            "fluent/fluent-bit:2.0.9"
        ],
        "RepoDigests": [
            "fluent/fluent-bit@sha256:d370ad358bc8c392d157ed89eaf4555d49041bf86b29a6a8150981d04d286be8"
        ],
        "Parent": "",
        "Comment": "buildkit.dockerfile.v0",
        "Created": "2023-02-03T16:04:31.653983161Z",
        "Container": "",
        "ContainerConfig": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": null,
            "Cmd": null,
            "Image": "",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": null
        },
        "DockerVersion": "",
        "Author": "Bazel",
        "Config": {
            "Hostname": "",
            "Domainname": "",
            "User": "0",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "2020/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt",
                "FLUENT_BIT_VERSION=2.0.9"
            ],
            "Cmd": [
                "/fluent-bit/bin/fluent-bit",
                "-c",
                "/fluent-bit/etc/fluent-bit.conf"
            ],
            "ArgsEscaped": true,
            "Image": "",
            "Volumes": null,
            "WorkingDir": "/",
            "Entrypoint": [
                "/fluent-bit/bin/fluent-bit"
            ],
            "OnBuild": null,
            "Labels": {
                "author": "Eduardo Silva <eduardo@calyptia.com>",
                "description": "Fluent Bit multi-architecture container image",
                "org.opencontainers.image.authors": "Eduardo Silva <eduardo@calyptia.com>",
                "org.opencontainers.image.created": "2023-02-03T14:43:34.440Z",
                "org.opencontainers.image.description": "Fast and Lightweight Logs and Metrics processor for Linux, BSD, OSX and Windows",
                "org.opencontainers.image.documentation": "https://docs.fluentbit.io/",
                "org.opencontainers.image.licenses": "Apache-2.0",
                "org.opencontainers.image.revision": "4c0ca4fc5f28a34ca7d80e18b0d529bf9d07b1c2",
                "org.opencontainers.image.source": "https://github.com/fluent/fluent-bit",
                "org.opencontainers.image.title": "fluent-bit",
                "org.opencontainers.image.url": "https://github.com/fluent/fluent-bit",
                "org.opencontainers.image.vendor": "Fluent Organization",
                "org.opencontainers.image.version": "2.0.9",
                "vendor": "Fluent Organization",
                "version": "2.0.9"
            }
        },
        "Architecture": "arm64",
        "Os": "linux",
        "Size": 80451506,
        "VirtualSize": 80451506,
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/containers/docker/overlay2/08ee6c6efa1c01c6ce3487cc74134f9bad2a2e86ff42d47c9e3f106828288001/diff:/var/lib/containers/docker/overlay2/201e87556e76bfdffd04c4e2500fab0f319e17020d7f2a8585b67d94cf57c2f8/diff:/var/lib/containers/docker/overlay2/5bf8aba71a900b2275ad4fbb5eee3720a24e3ea0127f8f7c6600728822fc9b32/diff:/var/lib/containers/docker/overlay2/644c0b6c30b1416c41fa9b2baf383d581979917328f1f579a7a752fe66802771/diff:/var/lib/containers/docker/overlay2/fbc9b2076e8ba869ef5b3bab2aca64caa762e4d58ab5a0bb56bf9397d4c15105/diff",
                "MergedDir": "/var/lib/containers/docker/overlay2/a64e2612d3b0dbe6bc07c4631e89c341ec8559cfb463d33e1ef07e9bebd926c4/merged",
                "UpperDir": "/var/lib/containers/docker/overlay2/a64e2612d3b0dbe6bc07c4631e89c341ec8559cfb463d33e1ef07e9bebd926c4/diff",
                "WorkDir": "/var/lib/containers/docker/overlay2/a64e2612d3b0dbe6bc07c4631e89c341ec8559cfb463d33e1ef07e9bebd926c4/work"
            },
            "Name": "overlay2"
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:fba4381f2bb70de24aadfb483fb2b5254e81b0f8db1959526a1beec8370363bc",
                "sha256:96e320b34b5478d8b369ca43ffaa88ff6dd9499ec72b792ca21b1e8b0c55670f",
                "sha256:7e2cf97ed8c49989106a276ec387e30bbd564b3c1a8ff943a13fd0f3cb4ad4f9",
                "sha256:09163696f0f9de226b0d3f27fa78ae14fff8ae1d4131e7804f5f47859e133f8c",
                "sha256:0a8123fc64c303f94a3279ec74c6ace8eb7cbfec86620d888b5b53e2ada6cad0",
                "sha256:931d784870af82afb151c63d54d972b545a825fe95ba39365661f711226a2993"
            ]
        },
        "Metadata": {
            "LastTagTime": "0001-01-01T00:00:00Z"
        }
    }
]
  • run an container:
[root@ ~]# docker run -ti docker.io/fluent/fluent-bit:2.0.9
<jemalloc>: Unsupported system page size
<jemalloc>: Unsupported system page size
.......
<jemalloc>: Unsupported system page size
Error in GnuTLS initialization: ASN1 parser: Element was not found.
<jemalloc>: Unsupported system page size
<jemalloc>: Unsupported system page size
<jemalloc>: Unsupported system page size
malloc: Cannot allocate memory
  • linux pagesize:
[root@host-192-168-10-107 ~]# getconf PAGESIZE
65536
  • linux arch
[root@host-192-168-10-107 ~]# uname -a
Linux host-192-168-10-107 4.19.90.S001.64.200410 #1 SMP Fri Aug 6 15:52:06 CST 2021 aarch64 aarch64 aarch64 GNU/Linux

fluentbit 2.0.5

[root@host-192-168-10-107 ~]# docker run -ti docker.io/fluent/fluent-bit:2.0.5
Unable to find image 'fluent/fluent-bit:2.0.5' locally
2.0.5: Pulling from fluent/fluent-bit
76c00402984b: Pull complete
a839fcbfad1b: Pull complete
4d704a88b82e: Pull complete
4c804370f3f0: Pull complete
18e8e7c376df: Pull complete
05afd198f5a3: Pull complete
Digest: sha256:c62fe1b22d17eff9eaf74eb5c67058e7f41bd9deeae16a57009f35f24bda03b9
Status: Downloaded newer image for fluent/fluent-bit:2.0.5
Fluent Bit v2.0.5
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2023/03/01 13:04:48] [ info] [fluent bit] version=2.0.5, commit=8e5f9f9106, pid=1
[2023/03/01 13:04:48] [ info] [storage] ver=1.3.0, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2023/03/01 13:04:48] [ info] [cmetrics] version=0.5.7
[2023/03/01 13:04:48] [ info] [ctraces ] version=0.2.5
[2023/03/01 13:04:48] [ info] [input:cpu:cpu.0] initializing
[2023/03/01 13:04:48] [ info] [input:cpu:cpu.0] storage_strategy='memory' (memory only)
[2023/03/01 13:04:48] [ info] [sp] stream processor started
[2023/03/01 13:04:48] [ info] [output:stdout:stdout.0] worker #0 started
[0] cpu.local: [1677675889.271466505, {"cpu_p"=>1.000000, "user_p"=>1.000000, "system_p"=>0.000000, "cpu0.p_cpu"=>1.000000, "cpu0.p_user"=>1.000000, "cpu0.p_system"=>0.000000}]
[0] cpu.local: [1677675890.271461753, {"cpu_p"=>0.000000, "user_p"=>0.000000, "system_p"=>0.000000, "cpu0.p_cpu"=>0.000000, "cpu0.p_user"=>0.000000, "cpu0.p_system"=>0.000000}]
[0] cpu.local: [1677675891.271456760, {"cpu_p"=>0.000000, "user_p"=>0.000000, "system_p"=>0.000000, "cpu0.p_cpu"=>0.000000, "cpu0.p_user"=>0.000000, "cpu0.p_system"=>0.000000}]
^C[2023/03/01 13:04:52] [engine] caught signal (SIGINT)
[2023/03/01 13:04:52] [ warn] [engine] service will shutdown in max 5 seconds
[2023/03/01 13:04:52] [ info] [input] pausing cpu.0
[0] cpu.local: [1677675892.271470877, {"cpu_p"=>0.000000, "user_p"=>0.000000, "system_p"=>0.000000, "cpu0.p_cpu"=>0.000000, "cpu0.p_user"=>0.000000, "cpu0.p_system"=>0.000000}]
[2023/03/01 13:04:53] [ info] [engine] service has stopped (0 pending tasks)
[2023/03/01 13:04:53] [ info] [input] pausing cpu.0
[2023/03/01 13:04:53] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2023/03/01 13:04:53] [ info] [output:stdout:stdout.0] thread worker #0 stopped
  • image metadata
[root@host-192-168-10-107 ~]# docker inspect fluent/fluent-bit:2.0.5
[
    {
        "Id": "sha256:24b654cc07810912ccef87ecd36ce8d850de94917de26822b09a904fa4cae777",
        "RepoTags": [
            "fluent/fluent-bit:2.0.5"
        ],
        "RepoDigests": [
            "fluent/fluent-bit@sha256:c62fe1b22d17eff9eaf74eb5c67058e7f41bd9deeae16a57009f35f24bda03b9"
        ],
        "Parent": "",
        "Comment": "buildkit.dockerfile.v0",
        "Created": "2022-11-10T23:49:00.901148657Z",
        "Container": "",
        "ContainerConfig": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": null,
            "Cmd": null,
            "Image": "",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": null
        },
        "DockerVersion": "",
        "Author": "Bazel",
        "Config": {
            "Hostname": "",
            "Domainname": "",
            "User": "0",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "2020/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt",
                "FLUENT_BIT_VERSION=2.0.5"
            ],
            "Cmd": [
                "/fluent-bit/bin/fluent-bit",
                "-c",
                "/fluent-bit/etc/fluent-bit.conf"
            ],
            "ArgsEscaped": true,
            "Image": "",
            "Volumes": null,
            "WorkingDir": "/",
            "Entrypoint": [
                "/fluent-bit/bin/fluent-bit"
            ],
            "OnBuild": null,
            "Labels": {
                "author": "Eduardo Silva <eduardo@calyptia.com>",
                "description": "Fluent Bit multi-architecture container image",
                "org.opencontainers.image.authors": "Eduardo Silva <eduardo@calyptia.com>",
                "org.opencontainers.image.created": "2022-11-10T22:22:09.014Z",
                "org.opencontainers.image.description": "Fast and Lightweight Logs and Metrics processor for Linux, BSD, OSX and Windows",
                "org.opencontainers.image.documentation": "https://docs.fluentbit.io/",
                "org.opencontainers.image.licenses": "Apache-2.0",
                "org.opencontainers.image.revision": "8e5f9f910694e059d692438d51d7dc3b46e63e18",
                "org.opencontainers.image.source": "https://github.com/fluent/fluent-bit",
                "org.opencontainers.image.title": "fluent-bit",
                "org.opencontainers.image.url": "https://github.com/fluent/fluent-bit",
                "org.opencontainers.image.vendor": "Fluent Organization",
                "org.opencontainers.image.version": "2.0.5",
                "vendor": "Fluent Organization",
                "version": "2.0.5"
            }
        },
        "Architecture": "arm64",
        "Os": "linux",
        "Size": 73264667,
        "VirtualSize": 73264667,
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/containers/docker/overlay2/791e8038c4bc4152c8e1cfe55be225fc1303b2664ab316b72b92cbf749eb7c12/diff:/var/lib/containers/docker/overlay2/83604d4b9e3686d19f8c27f70241ae1d3a7afea641420ae4a2a07e3ab932dc5a/diff:/var/lib/containers/docker/overlay2/8c778b17b65814c53251ac751099cda76b67f7483b110539c24ba25d26abd55a/diff:/var/lib/containers/docker/overlay2/542582daaa618d9bfb16ee3e2c312f3e29ae0c174c2b17ee0ee46bc29b0c8c77/diff:/var/lib/containers/docker/overlay2/ab450c5f80b8a8faae68771fcb4db88ccd012b805067e05e4860b853709d2327/diff",
                "MergedDir": "/var/lib/containers/docker/overlay2/0b917067f6c24a0812327e60557cd2347e261aa3bf19a3f0ad8c61db7f97e1e7/merged",
                "UpperDir": "/var/lib/containers/docker/overlay2/0b917067f6c24a0812327e60557cd2347e261aa3bf19a3f0ad8c61db7f97e1e7/diff",
                "WorkDir": "/var/lib/containers/docker/overlay2/0b917067f6c24a0812327e60557cd2347e261aa3bf19a3f0ad8c61db7f97e1e7/work"
            },
            "Name": "overlay2"
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:c306578afebba72fc207e8c8d93548f2eebae355c8a8c6a259db68c4edd79413",
                "sha256:96e320b34b5478d8b369ca43ffaa88ff6dd9499ec72b792ca21b1e8b0c55670f",
                "sha256:7e2cf97ed8c49989106a276ec387e30bbd564b3c1a8ff943a13fd0f3cb4ad4f9",
                "sha256:5e087d956f3e62bd034dd0712bc4cbef8fda55fba0b11a7d0564f294887c7079",
                "sha256:32a2f5bc81c320fde5b179654a84b72af7e1764082d5143df1cbfd7d5ee883f9",
                "sha256:f9215e00f5ee2579cb74912b45db4bb28a0255a94ea48dbf5eb94a609c3e71a6"
            ]
        },
        "Metadata": {
            "LastTagTime": "0001-01-01T00:00:00Z"
        }
    }
]
+1
JaredTan95 on Mar 1, 2023

I have the same issue, My Linux page size is 65535: 企业微信截图_3ef11479-41c9-48bc-b49c-dbb6ca49ed6d

I upgrade fluent bit from 2.0.5 to 2.0.9. But 2.0.5 works well. That’s wired. @patrick-stephens FYI

+1
JaredTan95 on Feb 24, 2023
Read more comments on GitHub
← fluent-bit: Official Windows build for 1.6.1 does not support built-in HTTP server
fluent-bit: out_cloudwatch: Segmentation fault →