ClickHouse: Clickhouse on ARM spams libunwind: Unsupported .eh_frame_hdr version to the stdout

Describe the bug ARM build of Clickhouse prints a ton of libunwind: Unsupported .eh_frame_hdr version

How to reproduce

Checkout 20.6.7.4 version
./configure && make && make install librdkafka shipped with Clickhouse (couldn’t get it in on ARM otherwise)
Prepare the build CC=clang-8 CXX=clang++-8 cmake -G Ninja -DCPPKAFKA_CMAKE_VERBOSE=ON -DENABLE_RDKAFKA=ON -DUSE_INTERNAL_RDKAFKA_LIBRARY=OFF -DRDKAFKA_INCLUDE_DIR=/usr/local/include/ .. (using gcc leads to linker error unless shared build is chosen; the issue reproduces with shared build though)
Build Clickhouse binary using ninja
Wrap the binary up into a Docker image and start it on an ARM server

Expected behavior The same x86 version (taken from official Docker Hub) does not produce these errors – thus expect the ARM version to be clear of the messages as well.

Additional context The problem seems to be happening in production environment (more load, more queries) more often. As a side note, we noticed that sometimes Clickhouse stops ingesting events from Kafka without anything logged to normal / error log – the only difference we noticed is the mentioned message in the stdout; thus, assuming the correlation but cannot prove direct connection. The restart of the server resumes the ingest.

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 3
Comments: 22 (9 by maintainers)

Commits related to this issue

fix(dev): ClickHouse dev container on arm64 has errors In #29081, we changed the arm ClickHouse container at the last moment without realizing that the older version has this `libunwind: Unsupported ... — committed to getsentry/sentry by armenzg 3 years ago
fix(dev): ClickHouse dev container on arm64 has errors (#29284) In #29081, we changed the arm ClickHouse container at the last moment without realizing that the older version has this `libunwind: Un... — committed to getsentry/sentry by armenzg 3 years ago

Most upvoted comments

https://github.com/ClickHouse/ClickHouse/pull/51436

KochetovNicolai on Jun 26, 2023

Was able to reproduce it with a simple test:

-- ClickHouse client version 22.5.1.2079 (official build).

CREATE TABLE arm_unsupported_eh_frame_hdr_on_client
(
    `t` DateTime,
    `id` UInt64
)
ENGINE = MergeTree
PARTITION BY toYYYYMM(t)
ORDER BY id;

INSERT INTO arm_unsupported_eh_frame_hdr_on_client VALUES (now(), rand());

libunwind: Unsupported .eh_frame_hdr version
libunwind: Unsupported .eh_frame_hdr version

It looks like now it’s reported by the client. It I remove function calls from the values section (or will do insert … select instead, which is executed on the server) - the problem goes away.

filimonov on May 26, 2022

Update: seems to be related to query profiler - disabling it via setting query_profiler_real_time_period_ns / query_profiler_cpu_time_period_ns to 0 makes the messages go away.

How can I disable it?

armenzg on Oct 19, 2021

Hey guys! Just pulled official ARM image from clickhouse/clickhouse-server:22.3.5 and the spammy logs are back despite having query profiler disabled – @paskal maybe you know other settings to try out? 😃

yard on May 5, 2022

Reproduces on 23.5.3.24

myrrc on Jul 12, 2023

I just hit this in AWS when changing from a c6g.4xlarge to a r7g.2xlarge which is going from Graviton 2 to Graviton 3. I’ve been using docker.io/clickhouse/clickhouse-server:22.8.15.23-alpine@sha256:848b8290576a6ec36c29d04faae1130d29002342bc202b07dcd7143512a2fc77 which should have the patch based on what I could tell.

I also tried 23.2.4.12-alpine and the ubuntu variant which should also have an updated libunwind, but they also log the error.

Edit: after more testing, it’s also happening on the r6g.2xlarge, so not a Graviton2 vs Graviton3 issue it seems.

chancez on Mar 27, 2023

Yep seems so:

SELECT
    name,
    value
FROM system.settings
WHERE name LIKE '%profiler%'

Query id: 290d4e43-9088-4a87-95b2-29a19f1fbd16

┌─name───────────────────────────────┬─value───┐
│ query_profiler_real_time_period_ns │ 0       │
│ query_profiler_cpu_time_period_ns  │ 0       │
│ memory_profiler_step               │ 4194304 │
│ memory_profiler_sample_probability │ 0       │
└────────────────────────────────────┴─────────┘

yard on May 6, 2022