ClickHouse: DB::Exception: Memory limit (total) exceeded: While executing MergeTreeThread. (MEMORY_LIMIT_EXCEEDED)

Hello, My Friends,

I recently had a problem with a memory limit. The error information is as follows:

DB::Exception: Memory limit (total) exceeded: would use 225.69 GiB (attempt to allocate chunk of 4258026 bytes), maximum: 225.60 GiB. OvercommitTracker decision: Memory overcommit isn't used. Waiting time or overcommit denominator are set to zero.: (while reading column ue_pdn_conn_delay_msel): (while reading from part /data/d3d817c3-6548-4f27-95cf-8f13a4ce14de/1552653000_256_268_1/ from mark 0 with max_rows_to_read = 8192): While executing MergeTreeThread. (MEMORY_LIMIT_EXCEEDED)

or in the AggregatingTransform, JoiningTransform, MergeTreeThread phase,It’s all possible

The Current ClickHouse version 22.8.11.15

I tried to compare the latest version(23.3.2.37) and had the same problem, but 21.8.8.29 did not have this problem.

Recurring Scenarios:

I’m trying to do an associative query of two tables (A left join B): 1、Table A contains 150 fields and 10000000 records. 2、Table B contains 60 fields, including 800000 records. 3、Start 10 query processes to query data at the same time. 4、After all the queries are complete, start process 10 again to query the information at the same time. In this case, the preceding error information is returned. 5、In the process, I tried to modify memory_overcommit_ratio_denominator_for_user to remove the limitations of Memory Tracker. But it is not effect; 6、Also, each of my queries took about 30+ seconds.

I’ve also looked up all the issues, and it’s kind of like this one, #49654

So, I don’t know if this is a new feature or a bug.

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 24 (12 by maintainers)

Most upvoted comments

Did you say you found problems in this 21.8.15.7 version? But I didn’t reproduce it in 21.8.8.29

IMHO, you were just lucky with 21.8, and the issue did not reproduce with your query pattern and started to reproduce with the newer versions. You can check 21.8.8 using my approach, I believe it will reproduce.


Disclaimer: I want to clarify that I am just an ordinary person on the Internet. I have no connection or association with Clickhouse, Inc. or its development team. It is important to note that my opinion holds no value and should not be considered.