ClickHouse: Background Replicated Merges seems to be stuck from time to time, greatly affecting insert latency

We have latency-critical application (real-time statistics system with watchdogs), which is sensitive to insert latency. We’d like latency to be less than 5 seconds, or better. In production, we’ve optimized table structure and amount of shards so that average insert delay is 1-2 seconds, but sometimes insert can take 20 or even 300 seconds (!), which triggers all kinds of bad things in our system.

We have simplified our system till we’ve come to the following test stand:

We have clickhouse cluster, which has 3 replicas. Machines are hardware (not virtual) have both SSDs and HDDs, but in test described below all data parts reside on SSDs (HDD is for historic data to be used in future). Clickhouse version is 21.7.4.18

We have incoming table with engine null, single ReplicatedMergeTree table and single mat view to insert from incoming into replicated table. On each replica, there is a process making 1 insert per second via HTTP in RowBinary format of chunk with size of roughly 50 megabytes, we allow up to 4 inserts in parallel, if all 4 inserts did no finish when next second data chunk arrives, it is discarded. There is no SELECTs during test, except occasional manual select from system.events_log system.parts_log, etc. We have separate ZK cluster of 3 machines set up for this cluster. It was set up empty, also clickhouse was set up empty. So we’ve started from the scratch.

Usually insert takes 500-1000ms. But sometimes several inserts can take 20 or even 300 seconds. We have noticed this always coincides with long running merge.

We have set both parts_to_delay_insert and parts_to_throw_insert to 150 so we will get an exception in case # of parts increases beyond this. But we get 0 exceptions.

We have increased max_replicated_merges_in_queue to 32 and decreased max_bytes_to_merge_at_max_space_in_pool to 16GB in attempt to limit time of longest merges and to allow more merges, this helped reduce maximum slowdown times from 300 to perhaps 100 seconds. our CPU usage is on the order of 10-15%, CPU has 48 cores. Table of system.merges contains 3-5 entries on average, never seen more than 10 entries.

Slowdowns can happen randomly on any single machine, or they can sometimes happen together on 2 or 3 machines together.

Slighly before slowdown the following entries start to appear in log

2021.07.23 17:45:05.689957 [ 19267 ] {} <Warning> default.statshouse_value_1s (ReplicatedMergeTreePartCheckThread): Checking part 1627030800_61998_62022_2 2021.07.23 17:45:05.689992 [ 19325 ] {} <Information> default.statshouse_value_1s (cfd8e29c-644b-4ef8-8fd8-e29c644beef8): DB::Exception: No active replica has part 1627030800_61998_62022_2 or covering part 2021.07.23 17:45:05.690146 [ 19267 ] {} <Warning> default.statshouse_value_1s (ReplicatedMergeTreePartCheckThread): Checking if anyone has a part 1627030800_61998_62022_2 or covering part. 2021.07.23 17:45:05.692051 [ 19267 ] {} <Warning> default.statshouse_value_1s (ReplicatedMergeTreePartCheckThread): Found parts with the same min block and with the same max block as the missing part 1627030800_61998_62022_2. Hoping that it will eventually appear as a result of a merge.

Also the following errors start to appear - hundreds similar entries per second. 2021.07.23 17:45:09.547801 [ 19422 ] {} <Information> default.statshouse_value_1s (ReplicatedMergeTreeQueue): Not executing log entry queue-0000087634 for part 1627030800_62014_62018_1 because another log entry for the same part is being processed. This shouldn't happen often. 2021.07.23 17:45:09.547825 [ 19422 ] {} <Information> default.statshouse_value_1s (cfd8e29c-644b-4ef8-8fd8-e29c644beef8): Will not fetch part 1627030800_62014_62018_1 covering 1627030800_62017_62017_0. Not executing log entry queue-0000087634 for part 1627030800_62014_62018_1 because another log entry for the same part is being processed. This shouldn't happen often. 2021.07.23 17:45:09.547854 [ 19267 ] {} <Information> default.statshouse_value_1s (ReplicatedMergeTreeQueue): Not executing log entry queue-0000087636 for part 1627030800_62014_62018_1 because another log entry for the same part is being processed. This shouldn't happen often.

Also the following entries start to appear

2021.07.23 18:26:46.850539 [ 19322 ] {} <Information> default.statshouse_value_1s (cfd8e29c-644b-4ef8-8fd8-e29c644beef8): The part /clickhouse/tables/0/statshouse_value_1s/replicas/ch846209/parts/1627052400_4794_4799_1 on a replica suddenly appeared, will recheck checksums 2021.07.23 18:26:50.609477 [ 19421 ] {} <Information> default.statshouse_value_1s (cfd8e29c-644b-4ef8-8fd8-e29c644beef8): The part /clickhouse/tables/0/statshouse_value_1s/replicas/ch846210/parts/1627052400_4805_4809_1 on a replica suddenly appeared, will recheck checksums 2021.07.23 18:27:57.784573 [ 19362 ] {} <Information> default.statshouse_value_1s (cfd8e29c-644b-4ef8-8fd8-e29c644beef8): The part /clickhouse/tables/0/statshouse_value_1s/replicas/ch846210/parts/1627052400_5006_5011_1 on a replica suddenly appeared, will recheck checksums 2021.07.23 18:30:36.799614 [ 19417 ] {} <Information> default.statshouse_value_1s (cfd8e29c-644b-4ef8-8fd8-e29c644beef8): The part /clickhouse/tables/0/statshouse_value_1s/replicas/ch846210/parts/1627052400_5484_5489_1 on a replica suddenly appeared, will recheck checksums ExaminingQueries.txt TableDefinitions.txt 230721.log Grafana monitroing system shows that around time of slowdown event the number of inodes slowly decrease than slowly increase by ~number of columns per second inserted, disk usage, disk read and disk writes show violent spikes, disk space slowly decreases then suddenly increases by 20-40 gigabytes.

All this seems like a bug.

Here is a link with ProfileEvents.Names and ProfileEvents.Values for typical and slow queries https://docs.google.com/spreadsheets/d/1lizbjRZ-DKMOYSDz19r1RGvcfoqrQHBDgeFpYJXcl8g/edit#gid=0

Attached is table definitions, queries examining machine state, clickhouse log.

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 3
  • Comments: 38 (32 by maintainers)

Most upvoted comments

Ok, increased sampling 1000x, clickhouse became little slower. New contender!

│ 2021-08-06 10:20:00 │    1199 │     0 │      0 │    70 │                          1214 │                   5896 │     │
│ 2021-08-06 10:30:00 │    1200 │     0 │      0 │    67 │                          1527 │                   6662 │     │
│ 2021-08-06 10:40:00 │    1110 │     9 │      5 │    63 │                          1748 │                  43744 │     │
│ 2021-08-06 10:50:00 │    1105 │    11 │      4 │    64 │                          1768 │                  36777 │     │
│ 2021-08-06 11:00:00 │    1143 │     5 │      0 │    65 │                          1727 │                  15693 │     │
│ 2021-08-06 11:10:00 │     769 │    58 │     19 │    36 │                          2799 │                 101216 │     │ <-- Slow down
│ 2021-08-06 11:20:00 │    1200 │     0 │      0 │    69 │                          1601 │                   5343 │     │
│ 2021-08-06 11:30:00 │    1201 │     0 │      0 │    69 │                          1658 │                   7124 │     │
│ 2021-08-06 11:40:00 │     839 │     0 │      0 │    48 │                          1656 │                   6358 │     │

Longest queries

┌─query_duration_ms─┬──────────event_time─┬─query_id─────────────────────────────┐
│            124183 │ 2021-08-06 12:08:43 │ aa2439fd-715f-4a3f-b27a-42b2264cb4af │
│            123233 │ 2021-08-06 12:08:43 │ 6e532159-e8b5-455e-aec9-c63cd9020a1f │
│            122409 │ 2021-08-06 12:08:44 │ 2776fb46-35b2-4b74-baf3-fabd8f186e46 │
│            122368 │ 2021-08-06 12:08:43 │ 02783684-9f29-472e-bd9b-5fefc9c09d94 │
│            101216 │ 2021-08-06 11:13:08 │ e9ebaa13-5a34-4e0a-9ed0-0ef51533758c │
│            100638 │ 2021-08-06 11:13:08 │ 1a9e164f-e2f8-4926-ac96-8abe78903f40 │
│             99752 │ 2021-08-06 11:13:10 │ cb6ad53d-2aaa-42d3-9771-70662112369e │
│             99457 │ 2021-08-06 11:13:08 │ 3acbe41b-2f0f-4851-808e-fc590b9cb46e │
│             95826 │ 2021-08-06 09:28:51 │ 9b510f3b-5dc5-4cbf-b7da-5f91cbef16f5 │
│             94763 │ 2021-08-06 09:28:51 │ 73b7538b-98ed-43ef-a50e-312cb8741b34 │
│             93758 │ 2021-08-06 09:28:51 │ d298a2c2-2fe5-4104-b00d-431f18a84a34 │
│             76569 │ 2021-08-06 09:28:31 │ 5aef5079-bbe9-43c3-8d9c-78ead9acc4b0 │
│             67416 │ 2021-08-06 12:08:47 │ ed78e145-e600-4484-988e-91af69a297d9 │

Now we have thousands of trace events per insert op.

SELECT arrayStringConcat(arrayMap(x -> demangle(addressToSymbol(x)), trace), '\n') AS trace_functions, count(*) as cc FROM system.trace_log WHERE query_id='e9ebaa13-5a34-4e0a-9ed0-0ef51533758c' group by trace order by cc desc limit 20
Result 
┌─trace_functions────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬────cc─┐
│ __lll_lock_wait
__GI___pthread_mutex_lock
std::__1::mutex::lock()
DB::MergeTreeData::renameTempPartAndAdd(std::__1::shared_ptr<DB::IMergeTreeDataPart>&, SimpleIncrement*, DB::MergeTreeData::Transaction*, DB::MergeTreeDeduplicationLog*)
DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)
DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&)
DB::PushingToViewsBlockOutputStream::write(DB::Block const&)
DB::AddingDefaultBlockOutputStream::write(DB::Block const&)
DB::SquashingBlockOutputStream::finalize()
DB::SquashingBlockOutputStream::writeSuffix()
DB::PushingToViewsBlockOutputStream::writeSuffix()
DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*)
DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>, std::__1::optional<DB::FormatSettings> const&)
DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&)
DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&)
DB::HTTPServerConnection::run()
Poco::Net::TCPServerConnection::start()
Poco::Net::TCPServerDispatcher::run()
Poco::PooledThread::run()
Poco::ThreadImpl::runnableEntry(void*)
start_thread
__clone │ 71127 │
│ DB::QueryProfilerReal::signalHandler(int, siginfo_t*, void*)
                                                                                                                                                                                               │   184 │
│ DB::ColumnVector<int>::insertFrom(DB::IColumn const&, unsigned long)
DB::AggregatingSortedAlgorithm::AggregatingMergedData::startGroup(std::__1::vector<DB::IColumn const*, std::__1::allocator<DB::IColumn const*> > const&, unsigned long)
DB::AggregatingSortedAlgorithm::merge()
DB::MergeTreeDataWriter::mergeBlock(DB::Block const&, std::__1::vector<DB::SortColumnDescription, std::__1::allocator<DB::SortColumnDescription> >, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 15ul, 16ul>*&)
DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context const>)
DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&)
DB::PushingToViewsBlockOutputStream::write(DB::Block const&)
DB::AddingDefaultBlockOutputStream::write(DB::Block const&)
DB::SquashingBlockOutputStream::finalize()
DB::SquashingBlockOutputStream::writeSuffix()
DB::PushingToViewsBlockOutputStream::writeSuffix()
DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*)
DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>, std::__1::optional<DB::FormatSettings> const&)
DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&)
DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&)
DB::HTTPServerConnection::run()
Poco::Net::TCPServerConnection::start()
Poco::Net::TCPServerDispatcher::run()
Poco::PooledThread::run()
Poco::ThreadImpl::runnableEntry(void*)
start_thread
__clone │   132 │
│ __pthread_cond_timedwait
std::__1::condition_variable::__do_timed_wait(std::__1::unique_lock<std::__1::mutex>&, std::__1::chrono::time_point<std::__1::chrono::system_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >)


zkutil::ZooKeeper::multiImpl(std::__1::vector<std::__1::shared_ptr<Coordination::Request>, std::__1::allocator<std::__1::shared_ptr<Coordination::Request> > > const&, std::__1::vector<std::__1::shared_ptr<Coordination::Response>, std::__1::allocator<std::__1::shared_ptr<Coordination::Response> > >&)
DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)
DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&)
DB::PushingToViewsBlockOutputStream::write(DB::Block const&)
DB::AddingDefaultBlockOutputStream::write(DB::Block const&)
DB::SquashingBlockOutputStream::finalize()
DB::SquashingBlockOutputStream::writeSuffix()
DB::PushingToViewsBlockOutputStream::writeSuffix()
DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*)
DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>, std::__1::optional<DB::FormatSettings> const&)
DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&)
DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&)
DB::HTTPServerConnection::run()
Poco::Net::TCPServerConnection::start()
Poco::Net::TCPServerDispatcher::run()
Poco::PooledThread::run()
Poco::ThreadImpl::runnableEntry(void*)
start_thread
__clone │   131 │
│ __pthread_cond_timedwait
std::__1::condition_variable::__do_timed_wait(std::__1::unique_lock<std::__1::mutex>&, std::__1::chrono::time_point<std::__1::chrono::system_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >)


zkutil::ZooKeeper::createImpl(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&)
zkutil::ZooKeeper::create(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int)
DB::EphemeralLockInZooKeeper::EphemeralLockInZooKeeper(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, zkutil::ZooKeeper&, std::__1::vector<std::__1::shared_ptr<Coordination::Request>, std::__1::allocator<std::__1::shared_ptr<Coordination::Request> > >*)
DB::StorageReplicatedMergeTree::allocateBlockNumber(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<zkutil::ZooKeeper> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const
DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)
DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&)
DB::PushingToViewsBlockOutputStream::write(DB::Block const&)
DB::AddingDefaultBlockOutputStream::write(DB::Block const&)
DB::SquashingBlockOutputStream::finalize()
DB::SquashingBlockOutputStream::writeSuffix()
DB::PushingToViewsBlockOutputStream::writeSuffix()
DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*)
DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>, std::__1::optional<DB::FormatSettings> const&)
DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&)
DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&)
DB::HTTPServerConnection::run()
Poco::Net::TCPServerConnection::start()
Poco::Net::TCPServerDispatcher::run()
Poco::PooledThread::run()
Poco::ThreadImpl::runnableEntry(void*)
start_thread
__clone │   114 │
│ __pthread_cond_timedwait
std::__1::condition_variable::__do_timed_wait(std::__1::unique_lock<std::__1::mutex>&, std::__1::chrono::time_point<std::__1::chrono::system_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >)


zkutil::ZooKeeper::createImpl(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&)
zkutil::ZooKeeper::create(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int)
DB::EphemeralLockInZooKeeper::EphemeralLockInZooKeeper(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, zkutil::ZooKeeper&, std::__1::vector<std::__1::shared_ptr<Coordination::Request>, std::__1::allocator<std::__1::shared_ptr<Coordination::Request> > >*)
DB::StorageReplicatedMergeTree::allocateBlockNumber(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<zkutil::ZooKeeper> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const
DB::ReplicatedMergeTreeBlockOutputStream::commitPart(std::__1::shared_ptr<zkutil::ZooKeeper>&, std::__1::shared_ptr<DB::IMergeTreeDataPart>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)
DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&)
DB::PushingToViewsBlockOutputStream::write(DB::Block const&)
DB::AddingDefaultBlockOutputStream::write(DB::Block const&)
DB::SquashingBlockOutputStream::finalize()
DB::SquashingBlockOutputStream::writeSuffix()
DB::PushingToViewsBlockOutputStream::writeSuffix()
DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*)
DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>, std::__1::optional<DB::FormatSettings> const&)
DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&)
DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&)
DB::HTTPServerConnection::run()
Poco::Net::TCPServerConnection::start()
Poco::Net::TCPServerDispatcher::run()
Poco::PooledThread::run()
Poco::ThreadImpl::runnableEntry(void*)
start_thread
__clone │    93 │
│ DB::AggregatingSortedAlgorithm::AggregatingMergedData::addRow(DB::SortCursor&)
DB::AggregatingSortedAlgorithm::merge()
DB::MergeTreeDataWriter::mergeBlock(DB::Block const&, std::__1::vector<DB::SortColumnDescription, std::__1::allocator<DB::SortColumnDescription> >, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 15ul, 16ul>*&)
DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context const>)
DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&)
DB::PushingToViewsBlockOutputStream::write(DB::Block const&)
DB::AddingDefaultBlockOutputStream::write(DB::Block const&)
DB::SquashingBlockOutputStream::finalize()
DB::SquashingBlockOutputStream::writeSuffix()
DB::PushingToViewsBlockOutputStream::writeSuffix()
DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*)
DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>, std::__1::optional<DB::FormatSettings> const&)
DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&)
DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&)
DB::HTTPServerConnection::run()
Poco::Net::TCPServerConnection::start()
Poco::Net::TCPServerDispatcher::run()
Poco::PooledThread::run()
Poco::ThreadImpl::runnableEntry(void*)
start_thread
__clone │    61 │
│ DB::ColumnVector<int>::compareAt(unsigned long, unsigned long, DB::IColumn const&, int) const
DB::AggregatingSortedAlgorithm::merge()
DB::MergeTreeDataWriter::mergeBlock(DB::Block const&, std::__1::vector<DB::SortColumnDescription, std::__1::allocator<DB::SortColumnDescription> >, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 15ul, 16ul>*&)
DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context const>)
DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&)
DB::PushingToViewsBlockOutputStream::write(DB::Block const&)
DB::AddingDefaultBlockOutputStream::write(DB::Block const&)
DB::SquashingBlockOutputStream::finalize()
DB::SquashingBlockOutputStream::writeSuffix()
DB::PushingToViewsBlockOutputStream::writeSuffix()
DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*)
DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>, std::__1::optional<DB::FormatSettings> const&)
DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&)
DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&)
DB::HTTPServerConnection::run()
Poco::Net::TCPServerConnection::start()
Poco::Net::TCPServerDispatcher::run()
Poco::PooledThread::run()
Poco::ThreadImpl::runnableEntry(void*)
start_thread
__clone │    52 │
│ DB::AggregateFunctionQuantile<float, DB::QuantileTDigest<float>, DB::NameQuantilesTDigest, false, float, true>::merge(char*, char const*, DB::Arena*) const
DB::AggregatingSortedAlgorithm::AggregatingMergedData::addRow(DB::SortCursor&)
DB::AggregatingSortedAlgorithm::merge()
DB::MergeTreeDataWriter::mergeBlock(DB::Block const&, std::__1::vector<DB::SortColumnDescription, std::__1::allocator<DB::SortColumnDescription> >, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 15ul, 16ul>*&)
DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context const>)
DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&)
DB::PushingToViewsBlockOutputStream::write(DB::Block const&)
DB::AddingDefaultBlockOutputStream::write(DB::Block const&)
DB::SquashingBlockOutputStream::finalize()
DB::SquashingBlockOutputStream::writeSuffix()
DB::PushingToViewsBlockOutputStream::writeSuffix()
DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*)
DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>, std::__1::optional<DB::FormatSettings> const&)
DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&)
DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&)
DB::HTTPServerConnection::run()
Poco::Net::TCPServerConnection::start()
Poco::Net::TCPServerDispatcher::run()
Poco::PooledThread::run()
Poco::ThreadImpl::runnableEntry(void*)
start_thread
__clone │    52 │
│ void DB::Aggregator::executeImplBatch<false, DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > > >(DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >&, DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >::State&, DB::Arena*, unsigned long, DB::Aggregator::AggregateFunctionInstruction*, char*) const
void DB::Aggregator::executeImpl<DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > > >(DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >&, DB::Arena*, unsigned long, std::__1::vector<DB::IColumn const*, std::__1::allocator<DB::IColumn const*> >&, DB::Aggregator::AggregateFunctionInstruction*, bool, char*) const
DB::Aggregator::executeOnBlock(std::__1::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn> > >, unsigned long, DB::AggregatedDataVariants&, std::__1::vector<DB::IColumn const*, std::__1::allocator<DB::IColumn const*> >&, std::__1::vector<std::__1::vector<DB::IColumn const*, std::__1::allocator<DB::IColumn const*> >, std::__1::allocator<std::__1::vector<DB::IColumn const*, std::__1::allocator<DB::IColumn const*> > > >&, bool&)
DB::AggregatingTransform::consume(DB::Chunk)
DB::AggregatingTransform::work()

DB::PipelineExecutor::executeStepImpl(unsigned long, unsigned long, std::__1::atomic<bool>*)
DB::PipelineExecutor::executeStep(std::__1::atomic<bool>*)
DB::PullingPipelineExecutor::pull(DB::Chunk&)
DB::PullingPipelineExecutor::pull(DB::Block&)
DB::PipelineExecutingBlockInputStream::readImpl()
DB::IBlockInputStream::read()
DB::MaterializingBlockInputStream::readImpl()
DB::IBlockInputStream::read()
DB::SquashingBlockInputStream::readImpl()
DB::IBlockInputStream::read()
DB::ConvertingBlockInputStream::readImpl()
DB::IBlockInputStream::read()
DB::PushingToViewsBlockOutputStream::process(DB::Block const&, DB::PushingToViewsBlockOutputStream::ViewInfo&)
DB::PushingToViewsBlockOutputStream::write(DB::Block const&)
DB::AddingDefaultBlockOutputStream::write(DB::Block const&)
DB::SquashingBlockOutputStream::finalize()
DB::SquashingBlockOutputStream::writeSuffix()
DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*)
DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>, std::__1::optional<DB::FormatSettings> const&)
DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&)
DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&)
DB::HTTPServerConnection::run()
Poco::Net::TCPServerConnection::start()
Poco::Net::TCPServerDispatcher::run()
Poco::PooledThread::run()
Poco::ThreadImpl::runnableEntry(void*)
start_thread
__clone │    51 │
│ DB::ColumnVector<int>::deserializeAndInsertFromArena(char const*)
void HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> >::forEachValue<void DB::Aggregator::convertToBlockImplFinal<DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >, HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >(DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >&, HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> >&, std::__1::vector<DB::IColumn*, std::__1::allocator<DB::IColumn*> >, std::__1::vector<COW<DB::IColumn>::mutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, DB::Arena*) const::'lambda'(DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > > const&, HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> >&)>(DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >&&)
void DB::Aggregator::convertToBlockImplFinal<DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >, HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >(DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >&, HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> >&, std::__1::vector<DB::IColumn*, std::__1::allocator<DB::IColumn*> >, std::__1::vector<COW<DB::IColumn>::mutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, DB::Arena*) const
void DB::Aggregator::convertToBlockImpl<DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >, HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >(DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >&, HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> >&, std::__1::vector<COW<DB::IColumn>::mutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, std::__1::vector<DB::PODArray<char*, 4096ul, Allocator<false, false>, 15ul, 16ul>*, std::__1::allocator<DB::PODArray<char*, 4096ul, Allocator<false, false>, 15ul, 16ul>*> >&, std::__1::vector<COW<DB::IColumn>::mutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, DB::Arena*, bool) const
DB::Aggregator::prepareBlockAndFillSingleLevel(DB::AggregatedDataVariants&, bool) const
DB::ConvertingAggregatedToChunksTransform::mergeSingleLevel()

DB::PipelineExecutor::executeStepImpl(unsigned long, unsigned long, std::__1::atomic<bool>*)
DB::PipelineExecutor::executeStep(std::__1::atomic<bool>*)
DB::PullingPipelineExecutor::pull(DB::Chunk&)
DB::PullingPipelineExecutor::pull(DB::Block&)
DB::PipelineExecutingBlockInputStream::readImpl()
DB::IBlockInputStream::read()
DB::MaterializingBlockInputStream::readImpl()
DB::IBlockInputStream::read()
DB::SquashingBlockInputStream::readImpl()
DB::IBlockInputStream::read()
DB::ConvertingBlockInputStream::readImpl()
DB::IBlockInputStream::read()
DB::PushingToViewsBlockOutputStream::process(DB::Block const&, DB::PushingToViewsBlockOutputStream::ViewInfo&)
DB::PushingToViewsBlockOutputStream::write(DB::Block const&)
DB::AddingDefaultBlockOutputStream::write(DB::Block const&)
DB::SquashingBlockOutputStream::finalize()
DB::SquashingBlockOutputStream::writeSuffix()
DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*)
DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>, std::__1::optional<DB::FormatSettings> const&)
DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&)
DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&)
DB::HTTPServerConnection::run()
Poco::Net::TCPServerConnection::start()
Poco::Net::TCPServerDispatcher::run()
Poco::PooledThread::run()
Poco::ThreadImpl::runnableEntry(void*)
start_thread
__clone │    46 │
│ DB::IAggregateFunctionHelper<DB::AggregateFunctionSum<double, double, DB::AggregateFunctionSumData<double>, (DB::AggregateFunctionSumType)0> >::addFree(DB::IAggregateFunction const*, char*, DB::IColumn const**, unsigned long, DB::Arena*)
DB::AggregatingSortedAlgorithm::AggregatingMergedData::addRow(DB::SortCursor&)
DB::AggregatingSortedAlgorithm::merge()
DB::MergeTreeDataWriter::mergeBlock(DB::Block const&, std::__1::vector<DB::SortColumnDescription, std::__1::allocator<DB::SortColumnDescription> >, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 15ul, 16ul>*&)
DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context const>)
DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&)
DB::PushingToViewsBlockOutputStream::write(DB::Block const&)
DB::AddingDefaultBlockOutputStream::write(DB::Block const&)
DB::SquashingBlockOutputStream::finalize()
DB::SquashingBlockOutputStream::writeSuffix()
DB::PushingToViewsBlockOutputStream::writeSuffix()
DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*)
DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>, std::__1::optional<DB::FormatSettings> const&)
DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&)
DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&)
DB::HTTPServerConnection::run()
Poco::Net::TCPServerConnection::start()
Poco::Net::TCPServerDispatcher::run()
Poco::PooledThread::run()
Poco::ThreadImpl::runnableEntry(void*)
start_thread
__clone │    41 │
│ UniquesHashSet<DefaultHash<unsigned long, void> >::merge(UniquesHashSet<DefaultHash<unsigned long, void> > const&)
DB::AggregatingSortedAlgorithm::AggregatingMergedData::addRow(DB::SortCursor&)
DB::AggregatingSortedAlgorithm::merge()
DB::MergeTreeDataWriter::mergeBlock(DB::Block const&, std::__1::vector<DB::SortColumnDescription, std::__1::allocator<DB::SortColumnDescription> >, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 15ul, 16ul>*&)
DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context const>)
DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&)
DB::PushingToViewsBlockOutputStream::write(DB::Block const&)
DB::AddingDefaultBlockOutputStream::write(DB::Block const&)
DB::SquashingBlockOutputStream::finalize()
DB::SquashingBlockOutputStream::writeSuffix()
DB::PushingToViewsBlockOutputStream::writeSuffix()
DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*)
DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>, std::__1::optional<DB::FormatSettings> const&)
DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&)
DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&)
DB::HTTPServerConnection::run()
Poco::Net::TCPServerConnection::start()
Poco::Net::TCPServerDispatcher::run()
Poco::PooledThread::run()
Poco::ThreadImpl::runnableEntry(void*)
start_thread
__clone │    29 │
│ DB::ColumnVector<int>::serializeValueIntoArena(unsigned long, DB::Arena&, char const*&) const
void DB::Aggregator::executeImplBatch<false, DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > > >(DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >&, DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >::State&, DB::Arena*, unsigned long, DB::Aggregator::AggregateFunctionInstruction*, char*) const
void DB::Aggregator::executeImpl<DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > > >(DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >&, DB::Arena*, unsigned long, std::__1::vector<DB::IColumn const*, std::__1::allocator<DB::IColumn const*> >&, DB::Aggregator::AggregateFunctionInstruction*, bool, char*) const
DB::Aggregator::executeOnBlock(std::__1::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn> > >, unsigned long, DB::AggregatedDataVariants&, std::__1::vector<DB::IColumn const*, std::__1::allocator<DB::IColumn const*> >&, std::__1::vector<std::__1::vector<DB::IColumn const*, std::__1::allocator<DB::IColumn const*> >, std::__1::allocator<std::__1::vector<DB::IColumn const*, std::__1::allocator<DB::IColumn const*> > > >&, bool&)
DB::AggregatingTransform::consume(DB::Chunk)
DB::AggregatingTransform::work()

DB::PipelineExecutor::executeStepImpl(unsigned long, unsigned long, std::__1::atomic<bool>*)
DB::PipelineExecutor::executeStep(std::__1::atomic<bool>*)
DB::PullingPipelineExecutor::pull(DB::Chunk&)
DB::PullingPipelineExecutor::pull(DB::Block&)
DB::PipelineExecutingBlockInputStream::readImpl()
DB::IBlockInputStream::read()
DB::MaterializingBlockInputStream::readImpl()
DB::IBlockInputStream::read()
DB::SquashingBlockInputStream::readImpl()
DB::IBlockInputStream::read()
DB::ConvertingBlockInputStream::readImpl()
DB::IBlockInputStream::read()
DB::PushingToViewsBlockOutputStream::process(DB::Block const&, DB::PushingToViewsBlockOutputStream::ViewInfo&)
DB::PushingToViewsBlockOutputStream::write(DB::Block const&)
DB::AddingDefaultBlockOutputStream::write(DB::Block const&)
DB::SquashingBlockOutputStream::finalize()
DB::SquashingBlockOutputStream::writeSuffix()
DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*)
DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>, std::__1::optional<DB::FormatSettings> const&)
DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&)
DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&)
DB::HTTPServerConnection::run()
Poco::Net::TCPServerConnection::start()
Poco::Net::TCPServerDispatcher::run()
Poco::PooledThread::run()
Poco::ThreadImpl::runnableEntry(void*)
start_thread
__clone │    28 │
│ UniquesHashSet<DefaultHash<unsigned long, void> >::merge(UniquesHashSet<DefaultHash<unsigned long, void> > const&)
DB::IAggregateFunctionHelper<DB::AggregateFunctionMerge>::addBatch(unsigned long, char**, unsigned long, DB::IColumn const**, DB::Arena*, long) const
void DB::Aggregator::executeImplBatch<false, DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > > >(DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >&, DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >::State&, DB::Arena*, unsigned long, DB::Aggregator::AggregateFunctionInstruction*, char*) const
void DB::Aggregator::executeImpl<DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > > >(DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >&, DB::Arena*, unsigned long, std::__1::vector<DB::IColumn const*, std::__1::allocator<DB::IColumn const*> >&, DB::Aggregator::AggregateFunctionInstruction*, bool, char*) const
DB::Aggregator::executeOnBlock(std::__1::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn> > >, unsigned long, DB::AggregatedDataVariants&, std::__1::vector<DB::IColumn const*, std::__1::allocator<DB::IColumn const*> >&, std::__1::vector<std::__1::vector<DB::IColumn const*, std::__1::allocator<DB::IColumn const*> >, std::__1::allocator<std::__1::vector<DB::IColumn const*, std::__1::allocator<DB::IColumn const*> > > >&, bool&)
DB::AggregatingTransform::consume(DB::Chunk)
DB::AggregatingTransform::work()

DB::PipelineExecutor::executeStepImpl(unsigned long, unsigned long, std::__1::atomic<bool>*)
DB::PipelineExecutor::executeStep(std::__1::atomic<bool>*)
DB::PullingPipelineExecutor::pull(DB::Chunk&)
DB::PullingPipelineExecutor::pull(DB::Block&)
DB::PipelineExecutingBlockInputStream::readImpl()
DB::IBlockInputStream::read()
DB::MaterializingBlockInputStream::readImpl()
DB::IBlockInputStream::read()
DB::SquashingBlockInputStream::readImpl()
DB::IBlockInputStream::read()
DB::ConvertingBlockInputStream::readImpl()
DB::IBlockInputStream::read()
DB::PushingToViewsBlockOutputStream::process(DB::Block const&, DB::PushingToViewsBlockOutputStream::ViewInfo&)
DB::PushingToViewsBlockOutputStream::write(DB::Block const&)
DB::AddingDefaultBlockOutputStream::write(DB::Block const&)
DB::SquashingBlockOutputStream::finalize()
DB::SquashingBlockOutputStream::writeSuffix()
DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*)
DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>, std::__1::optional<DB::FormatSettings> const&)
DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&)
DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&)
DB::HTTPServerConnection::run()
Poco::Net::TCPServerConnection::start()
Poco::Net::TCPServerDispatcher::run()
Poco::PooledThread::run()
Poco::ThreadImpl::runnableEntry(void*)
start_thread
__clone │    22 │
│ DB::ColumnVector<int>::deserializeAndInsertFromArena(char const*)
void HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> >::forEachValue<void DB::Aggregator::convertToBlockImplFinal<DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >, HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >(DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >&, HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> >&, std::__1::vector<DB::IColumn*, std::__1::allocator<DB::IColumn*> >, std::__1::vector<COW<DB::IColumn>::mutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, DB::Arena*) const::'lambda'(DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > > const&, HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> >&)>(DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >&&)
void DB::Aggregator::convertToBlockImplFinal<DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >, HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >(DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >&, HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> >&, std::__1::vector<DB::IColumn*, std::__1::allocator<DB::IColumn*> >, std::__1::vector<COW<DB::IColumn>::mutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, DB::Arena*) const
void DB::Aggregator::convertToBlockImpl<DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >, HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >(DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >&, HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> >&, std::__1::vector<COW<DB::IColumn>::mutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, std::__1::vector<DB::PODArray<char*, 4096ul, Allocator<false, false>, 15ul, 16ul>*, std::__1::allocator<DB::PODArray<char*, 4096ul, Allocator<false, false>, 15ul, 16ul>*> >&, std::__1::vector<COW<DB::IColumn>::mutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, DB::Arena*, bool) const
DB::Aggregator::prepareBlockAndFillSingleLevel(DB::AggregatedDataVariants&, bool) const
DB::ConvertingAggregatedToChunksTransform::mergeSingleLevel()

DB::PipelineExecutor::executeStepImpl(unsigned long, unsigned long, std::__1::atomic<bool>*)
DB::PipelineExecutor::executeStep(std::__1::atomic<bool>*)
DB::PullingPipelineExecutor::pull(DB::Chunk&)
DB::PullingPipelineExecutor::pull(DB::Block&)
DB::PipelineExecutingBlockInputStream::readImpl()
DB::IBlockInputStream::read()
DB::MaterializingBlockInputStream::readImpl()
DB::IBlockInputStream::read()
DB::SquashingBlockInputStream::readImpl()
DB::IBlockInputStream::read()
DB::ConvertingBlockInputStream::readImpl()
DB::IBlockInputStream::read()
DB::PushingToViewsBlockOutputStream::process(DB::Block const&, DB::PushingToViewsBlockOutputStream::ViewInfo&)
DB::PushingToViewsBlockOutputStream::write(DB::Block const&)
DB::AddingDefaultBlockOutputStream::write(DB::Block const&)
DB::SquashingBlockOutputStream::finalize()
DB::SquashingBlockOutputStream::writeSuffix()
DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*)
DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>, std::__1::optional<DB::FormatSettings> const&)
DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&)
DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&)
DB::HTTPServerConnection::run()
Poco::Net::TCPServerConnection::start()
Poco::Net::TCPServerDispatcher::run()
Poco::PooledThread::run()
Poco::ThreadImpl::runnableEntry(void*)
start_thread
__clone │    18 │
│ DB::AggregateFunctionArgMinMax<DB::AggregateFunctionArgMinMaxData<DB::SingleValueDataString, DB::AggregateFunctionMaxData<DB::SingleValueDataFixed<double> > > >::merge(char*, char const*, DB::Arena*) const
DB::AggregatingSortedAlgorithm::AggregatingMergedData::addRow(DB::SortCursor&)
DB::AggregatingSortedAlgorithm::merge()
DB::MergeTreeDataWriter::mergeBlock(DB::Block const&, std::__1::vector<DB::SortColumnDescription, std::__1::allocator<DB::SortColumnDescription> >, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 15ul, 16ul>*&)
DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context const>)
DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&)
DB::PushingToViewsBlockOutputStream::write(DB::Block const&)
DB::AddingDefaultBlockOutputStream::write(DB::Block const&)
DB::SquashingBlockOutputStream::finalize()
DB::SquashingBlockOutputStream::writeSuffix()
DB::PushingToViewsBlockOutputStream::writeSuffix()
DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*)
DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>, std::__1::optional<DB::FormatSettings> const&)
DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&)
DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&)
DB::HTTPServerConnection::run()
Poco::Net::TCPServerConnection::start()
Poco::Net::TCPServerDispatcher::run()
Poco::PooledThread::run()
Poco::ThreadImpl::runnableEntry(void*)
start_thread
__clone │    15 │
│ DB::ColumnString::insertFrom(DB::IColumn const&, unsigned long)
DB::AggregatingSortedAlgorithm::AggregatingMergedData::startGroup(std::__1::vector<DB::IColumn const*, std::__1::allocator<DB::IColumn const*> > const&, unsigned long)
DB::AggregatingSortedAlgorithm::merge()
DB::MergeTreeDataWriter::mergeBlock(DB::Block const&, std::__1::vector<DB::SortColumnDescription, std::__1::allocator<DB::SortColumnDescription> >, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >&, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 15ul, 16ul>*&)
DB::MergeTreeDataWriter::writeTempPart(DB::BlockWithPartition&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::Context const>)
DB::ReplicatedMergeTreeBlockOutputStream::write(DB::Block const&)
DB::PushingToViewsBlockOutputStream::write(DB::Block const&)
DB::AddingDefaultBlockOutputStream::write(DB::Block const&)
DB::SquashingBlockOutputStream::finalize()
DB::SquashingBlockOutputStream::writeSuffix()
DB::PushingToViewsBlockOutputStream::writeSuffix()
DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*)
DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>, std::__1::optional<DB::FormatSettings> const&)
DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&)
DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&)
DB::HTTPServerConnection::run()
Poco::Net::TCPServerConnection::start()
Poco::Net::TCPServerDispatcher::run()
Poco::PooledThread::run()
Poco::ThreadImpl::runnableEntry(void*)
start_thread
__clone │    14 │
│ __madvise

extent_dalloc_wrapper



tcache_bin_flush_small
free_default
Allocator<true, true>::free(void*, unsigned long)
DB::IAggregateFunctionDataHelper<DB::AggregateFunctionUniqUniquesHashSetData, DB::AggregateFunctionUniq<long, DB::AggregateFunctionUniqUniquesHashSetData> >::destroy(char*) const
DB::ColumnAggregateFunction::~ColumnAggregateFunction()
DB::ColumnAggregateFunction::~ColumnAggregateFunction()
DB::Block::~Block()
DB::SquashingBlockOutputStream::finalize()
DB::SquashingBlockOutputStream::writeSuffix()
DB::PushingToViewsBlockOutputStream::writeSuffix()
DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*)
DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>, std::__1::optional<DB::FormatSettings> const&)
DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&)
DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&)
DB::HTTPServerConnection::run()
Poco::Net::TCPServerConnection::start()
Poco::Net::TCPServerDispatcher::run()
Poco::PooledThread::run()
Poco::ThreadImpl::runnableEntry(void*)
start_thread
__clone │    14 │
│ DB::IAggregateFunctionHelper<DB::AggregateFunctionArgMinMax<DB::AggregateFunctionArgMinMaxData<DB::SingleValueDataString, DB::AggregateFunctionMaxData<DB::SingleValueDataFixed<double> > > > >::addBatch(unsigned long, char**, unsigned long, DB::IColumn const**, DB::Arena*, long) const
void DB::Aggregator::executeImplBatch<false, DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > > >(DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >&, DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >::State&, DB::Arena*, unsigned long, DB::Aggregator::AggregateFunctionInstruction*, char*) const
void DB::Aggregator::executeImpl<DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > > >(DB::AggregationMethodSerialized<HashMapTable<StringRef, HashMapCellWithSavedHash<StringRef, char*, DefaultHash<StringRef, void>, HashTableNoState>, DefaultHash<StringRef, void>, HashTableGrower<8ul>, Allocator<true, true> > >&, DB::Arena*, unsigned long, std::__1::vector<DB::IColumn const*, std::__1::allocator<DB::IColumn const*> >&, DB::Aggregator::AggregateFunctionInstruction*, bool, char*) const
DB::Aggregator::executeOnBlock(std::__1::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn> > >, unsigned long, DB::AggregatedDataVariants&, std::__1::vector<DB::IColumn const*, std::__1::allocator<DB::IColumn const*> >&, std::__1::vector<std::__1::vector<DB::IColumn const*, std::__1::allocator<DB::IColumn const*> >, std::__1::allocator<std::__1::vector<DB::IColumn const*, std::__1::allocator<DB::IColumn const*> > > >&, bool&)
DB::AggregatingTransform::consume(DB::Chunk)
DB::AggregatingTransform::work()

DB::PipelineExecutor::executeStepImpl(unsigned long, unsigned long, std::__1::atomic<bool>*)
DB::PipelineExecutor::executeStep(std::__1::atomic<bool>*)
DB::PullingPipelineExecutor::pull(DB::Chunk&)
DB::PullingPipelineExecutor::pull(DB::Block&)
DB::PipelineExecutingBlockInputStream::readImpl()
DB::IBlockInputStream::read()
DB::MaterializingBlockInputStream::readImpl()
DB::IBlockInputStream::read()
DB::SquashingBlockInputStream::readImpl()
DB::IBlockInputStream::read()
DB::ConvertingBlockInputStream::readImpl()
DB::IBlockInputStream::read()
DB::PushingToViewsBlockOutputStream::process(DB::Block const&, DB::PushingToViewsBlockOutputStream::ViewInfo&)
DB::PushingToViewsBlockOutputStream::write(DB::Block const&)
DB::AddingDefaultBlockOutputStream::write(DB::Block const&)
DB::SquashingBlockOutputStream::finalize()
DB::SquashingBlockOutputStream::writeSuffix()
DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::__1::atomic<bool>*)
DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>, std::__1::optional<DB::FormatSettings> const&)
DB::HTTPHandler::processQuery(std::__1::shared_ptr<DB::Context>, DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&)
DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&)
DB::HTTPServerConnection::run()
Poco::Net::TCPServerConnection::start()
Poco::Net::TCPServerDispatcher::run()
Poco::PooledThread::run()
Poco::ThreadImpl::runnableEntry(void*)
start_thread
__clone │    13 │
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴───────┘

Wow 71000 out of 73462 samples are waiting the mutex in DB::MergeTreeData::renameTempPartAndAdd

Does this sound right?

+1
hrissan on Aug 9, 2021
Read more comments on GitHub
← ClickHouse: argMaxSimpleState((1,1)) crash
ClickHouse: Bad size of marks file on Index →