ClickHouse: Too many open files while using materialized view with kafka source

Greetings. I have Ubuntu 18.04 source system. Clickhouse server version - 18.12.17. Default config. Database structure creation:

create table a (EventDate Date) ENGINE MergeTree() partition by toYYYYMM(EventDate) order by (EventDate) settings index_granularity=8192;

create table k (EventDate Date) ENGINE=Kafka SETTINGS
  kafka_broker_list = 'localhost:9092,localhost:9093,localhost:9094',
  kafka_topic_list = 'a',
  kafka_group_name = 'group1',
  kafka_format = 'JSONEachRow';

create materialized view w to a as select * from k;

As soon, as materialized view was created, amount of file descriptors for clickhouse server process starts to grow each second to the limit of max open files. Error logs are empty. Usual log contains only debug ant trace info. However syslog contains following errors:

Sep 22 18:22:38 kernel: [25449.716131] netlink: 'TCPHandler': attribute type 1 has an invalid length.
Sep 22 18:22:38 kernel: [25449.729949] netlink: 'TCPHandler': attribute type 1 has an invalid length.
Sep 22 18:22:40 kernel: [25451.228134] netlink: 'ParalInputsProc': attribute type 1 has an invalid length.
Sep 22 18:22:40 kernel: [25451.228175] netlink: 'ParalInputsProc': attribute type 1 has an invalid length.
Sep 22 18:22:42 kernel: [25453.227433] netlink: 'ParalInputsProc': attribute type 1 has an invalid length.

Am I missing something? Any help would be much appreciated.

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 8
  • Comments: 22 (7 by maintainers)

Most upvoted comments

We are also seeing this issue. Clickhouse currently has 456,357 files open; they are mostly netlink sockets:

clickhous 5174 clickhouse *377u     sock                0,8       0t0 41551004 protocol: NETLINK
clickhous 5174 clickhouse *378u     sock                0,8       0t0 41551005 protocol: NETLINK
clickhous 5174 clickhouse *395u     sock                0,8       0t0 41551817 protocol: NETLINK
clickhous 5174 clickhouse *396u     sock                0,8       0t0 41551818 protocol: NETLINK

At this point the kafka table engine + MV is unusable. Our clickhouse version is 18.12.13 revision 54407. I’m happy to provide more info to help debug the problem.