ByConity: Why INSERT VALUES queries cannot be auto forwarded from server to workers?
Currently, in InterpreterInsertQuery.cpp, the logic of forwarding INSERT query to workers is as following:
......
StorageCnchMergeTree * cnch_merge_tree = dynamic_cast<StorageCnchMergeTree *>(table.get());
/// Directly forward the query to cnch worker if select or infile
if (getContext()->getServerType() == ServerType::cnch_server && (query.select || query.in_file) && cnch_merge_tree)
{
/// Handle the insert commit for insert select/infile case in cnch server.
BlockInputStreamPtr in = cnch_merge_tree->writeInWorker(query_ptr, metadata_snapshot, getContext());
bool enable_staging_area_for_write = settings.enable_staging_area_for_write;
if (const auto * cnch_table = dynamic_cast<const StorageCnchMergeTree *>(table.get());
cnch_table && metadata_snapshot->hasUniqueKey() && !enable_staging_area_for_write)
{
/// for unique table, insert select|infile is committed from worker side
res.in = std::move(in);
}
else
{
auto txn = getContext()->getCurrentTransaction();
txn->setMainTableUUID(table->getStorageUUID());
res.in = std::make_shared<TransactionWrapperBlockInputStream>(in, std::move(txn));
}
return res;
}
BlockOutputStreams out_streams;
......
My questions:
- Why we only forward INSERT SELECT or INSERT INFILE queries to workers here?
- Is the design of not forwarding INSERT VALUES queries functionally difficult to implement, or would it cause performance issues?
Thank you very much.
About this issue
- Original URL
- State: closed
- Created 9 months ago
- Comments: 24 (1 by maintainers)
I think setting this config in worker config might break some use case, like kafka insertion. For now you can try it and if it doesn’t break any use case that it is ok. In future we will try to remove this config
@dmthuc Ok, thanks for your guidance. I will try it first.