ClickHouse: Do we have some mechanism to ensure that subqueries sent from initial node won't be optimized by rewritten.

Use case Do we have some mechanism to ensure that subqueries sent from initial node won’t be optimized by rewritten. This would make sense to reduce the exceptions Not found column which happened when initial node didn’t optimize by rewritting ast but worker node did it.

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 15 (14 by maintainers)

Most upvoted comments

@vdimir I found out why you can’t reproduce. In lastest version of clickhouse, optimize will not be executed on non-initial query, see https://github.com/ClickHouse/ClickHouse/pull/28746/files introduced by @amosbird

taiyang-li on Mar 11, 2022

@vdimir You should test it in clickhouse cluster with some replicas deployed with old version before the pr which brought optimize_rewrite_sum_if_to_count_if, and some deployed with new version after that pr.

create table

CREATE TABLE default.table on cluster ck_cluster_preonline 
(

    `reportTime` UInt32,
    `appId` UInt32,
    `platform` UInt16,
    `firstIFrameTs` UInt16,
    `day` Date DEFAULT toDate(rtime),
    `rtime` DateTime DEFAULT toDateTime(reportTime)
)
ENGINE = ReplicatedMergeTree('/clickhouse/tables/{layer}-{shard}/test/table', '{replica}')
PARTITION BY toYYYYMMDD(day)
ORDER BY (rtime, reportTime, appId)
TTL day + toIntervalMonth(3)
SETTINGS index_granularity = 8192 


CREATE TABLE IF NOT EXISTS default.table_all on cluster ck_cluster_preonline AS default.table
ENGINE = Distributed(ck_cluster_preonline, default, table, rand())

insert some data
Execute query:

SELECT 
    toStartOfMinute(toDateTime(reportTime)) AS rtime, 
    sumIf(multiIf((firstIFrameTs > 0) AND (firstIFrameTs <= 100), 1, 0), (appId = 60) AND (platform IN (0, 1)) AND (firstIFrameTs > 0)) AS _in1000ms, 
    sumIf(1, (appId = 60) AND (platform IN (0, 1)) AND (firstIFrameTs > 0)) AS total_secondout
FROM default.table_all 
GROUP BY rtime

taiyang-li on Mar 7, 2022

I tried to add test https://github.com/ClickHouse/ClickHouse/pull/35049 but it’s not fail locally.

@taiyang-li do you have reproducible example?

vdimir on Mar 4, 2022