ClickHouse: Catboost clickhouse-library-bridge is not responding

@rschu1ze Thank you very much, after you answer, all become clear and understandable.

Logging start working, and problem become visible.

In our ch setup 9012 port used as interserver_http_port, and so clickhouse-library-bridge not worked because Address already in use:

root@glprod-clickhouse:~# tail -fn 1000 /var/log/clickhouse-server-ru/clickhouse-library-bridge.err.log
2022.12.02 09:25:55.509957 [ 1074 ] {} <Error> Application: Net Exception: Address already in use: 127.0.0.1:9012

It will be great, if in docs somewhere it was be mentioned on which port clickhouse-library-bridge started (and how to change it), as you mentioned here:

./clickhouse-library-bridge --http-port=9012 # 9012 is the default port on which the server tries to connect, can be changed if needed

---

And also, please include this breaking changes (modelEvaluate -> catboostEvaluate) in changelog.

@oleg-savko About logging: As far as I see, tags “clickhouse_bridge_log” and “clickhouse_bridge_level” must be embedded into tag “logger”. Perhaps that was the reason why you got no log output (cf. https://github.com/ClickHouse/ClickHouse/issues/43605#issuecomment-1326284160, https://github.com/ClickHouse/ClickHouse/issues/43605#issuecomment-1326309887)

<clickhouse>
    <logger>
        <library_bridge_log>/var/log/clickhouse-server/clickhouse-library-bridge.log</library_bridge_log>
        <library_bridge_errlog>/var/log/clickhouse-server/clickhouse-library-bridge.err.log</library_bridge_errlog>
        <library_bridge_level>trace</library_bridge_level>
    </logger>
</clickhouse>

Note that log file “clickhouse-catboost-bridge.log” and “clickhouse-catboost-bridge.err.log” trace events in the library-bridge process. So, another reason why these files don’t exist is that the binary was somehow not started. Actually, the first error (https://github.com/ClickHouse/ClickHouse/issues/43605#issue-1463145593) looked like such a situation: clickhouse-library-bridge is not responding.

As @kssenii mentioned, the catboost evaluation functionality was moved from the server process into the library-bridge process (#40897). The library-bridge binary is started on demand (if not running already) during the first catboostEvaluate() invocation, or more specifically: during the first handshake attempt with the library-bridge. It is also possible to start the library-bridge binary manual (ideally before starting the server):

$ ./clickhouse-library-bridge --http-port=9012  # 9012 is the default port on which the server tries to connect, can be changed if needed

I tried the repo (https://github.com/ClickHouse/ClickHouse/issues/43605#issuecomment-1327413740) and with some fiddling due to the Docker environment, I manged to make it work for me. Steps:

1. sudo docker pull clickhouse/clickhouse-server
2. sudo docker run --name some-clickhouse-server --ulimit nofile=262144:262144 -v /home/ubuntu/config.xml:/etc/clickhouse-server/config.xml -v /home/ubuntu/users.xml:/etc/clickhouser-server/users.xml -v /home/ubuntu/libcatboostmodel.so:/etc/clickhouse-server/libcatboostmodel.so -v /home/ubuntu/simple_model.bin:/tmp/simple_model.bin clickhouse/clickhouse-server
3. sudo docker exec -it some-clickhouse-server clickhouse-client and SELECT catboostEvaluate('/tmp/simple_model.bin', 1., 2., 3, 4, 5, 6, 7, 8, 9, 10, 11)

After step 3., sudo docker exec -it some-clickhouse-server bash and ps aux | grep click shows:

clickho+       1  2.1  0.5 2634516 394592 ?      Ssl  13:13   0:00 /usr/bin/clickhouse-server --config-file=/etc/clickhouse-server/config.xml                                                                                                                          
root         227  0.2  0.1 480852 99364 pts/0    Ss+  13:13   0:00 clickhouse-client                                                                                                                                                                                   
clickho+     235  0.2  0.0 370460 49180 ?        Sl   13:13   0:00 /usr/bin/clickhouse-library-bridge --http-port 9012 --listen-host 127.0.0.1 --http-timeout 180000000 --log-path /tmp/library-bridge.log --err-log-path /tmp/library-bridge.err.log --log-level trace
root         250  0.0  0.0   6308   652 pts/1    S+   13:13   0:00 grep --color=auto click                                                                                                                                                                             

which is as expected.

Notes about 2: My config.xml looks that:

<?xml version="1.0"?>
<clickhouse>

    <listen_host>localhost</listen_host>
    <path>/var/lib/clickhouse1/</path>
    <tcp_port>9000</tcp_port>
    <http_port>8123</http_port>
    <users_config>users.xml</users_config>

    <logger>
        <level>trace</level>
        <console>true</console>
        <library_bridge_log>/tmp/library-bridge.log</library_bridge_log>
        <library_bridge_errlog>/tmp/library-bridge.err.log</library_bridge_errlog>
        <library_bridge_level>trace</library_bridge_level>
    </logger>

    <catboost_lib_path>/etc/clickhouse-server/libcatboostmodel.so</catboost_lib_path>

</clickhouse>

user.xml looks like that:

<?xml version="1.0"?>
<clickhouse>

    <profiles>
        <default>
            <allow_unrestricted_reads_from_keeper>1</allow_unrestricted_reads_from_keeper>
        </default>
    </profiles>

    <users>

        <default>
            <password></password>
            <networks>
                <ip>127.0.0.1</ip>
            </networks>
            <profile>default</profile>
            <quota>default</quota>
            <access_management>1</access_management>
        </default>

    </users>

    <quotas>
        <default></default>
    </quotas>

</clickhouse>

Files libcatboost.sh and simple_model.bin were taken from this integration test.

I’ll check deeper but I can imagine it did not work for you because some pieces of the configuration were missing. Maybe I can improve the documentation .

@rschu1ze can you please take a look?

Steps to reproduce:

Dockerfile:

from clickhouse/clickhouse-server:22.10.3.27

RUN apt-get update \
      && apt-get install clickhouse-common-static=22.10.3.27

Execute query:

SELECT catboostEvaluate('/etc/clickhouse-server/models/catboost_model_1b9ab50eedbe41a5be64a3a224eda68c.cbm',
        ...

Server log:

2022.11.25 12:21:39.003667 [ 1093 ] {3d111fc7-1d29-41cb-88dd-52ba4071ba16} <Error> LibraryBridgeHelper: Code: 86. DB::Exception: Received error from remote server /catboost_ping. HTTP status code: 404 Not Found, body: There is no handle /catboost_ping

Use / or /ping for health checks.
Or /replicas_status for more sophisticated health checks.

Send queries from your program with POST method or GET /?query=...

Use clickhouse-client:

For interactive data analysis:
    clickhouse-client

For batch query processing:
    clickhouse-client --query='SELECT 1' > result
    clickhouse-client < query > result
. (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER), Stack trace (when copying this message, always include the lines below):

0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0xce3a81a in /usr/bin/clickhouse
1. DB::assertResponseIsOk(Poco::Net::HTTPRequest const&, Poco::Net::HTTPResponse&, std::__1::basic_istream<char, std::__1::char_traits<char> >&, bool) @ 0xd029970 in /usr/bin/clickhouse
2. ? @ 0xf4db9e7 in /usr/bin/clickhouse
3. ? @ 0xf4d734c in /usr/bin/clickhouse
4. ? @ 0xf4d5a81 in /usr/bin/clickhouse
5. ? @ 0xf4d3b58 in /usr/bin/clickhouse
6. DB::readString(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, DB::ReadBuffer&) @ 0xce93002 in /usr/bin/clickhouse
7. DB::CatBoostLibraryBridgeHelper::bridgeHandShake() @ 0x13462211 in /usr/bin/clickhouse
8. DB::IBridgeHelper::startBridgeSync() @ 0x1346b77a in /usr/bin/clickhouse
9. DB::CatBoostLibraryBridgeHelper::getTreeCount() @ 0x134639b6 in /usr/bin/clickhouse
10. ? @ 0x9ab58c7 in /usr/bin/clickhouse
11. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&) const @ 0x114675ea in /usr/bin/clickhouse
12. DB::IFunctionOverloadResolver::getReturnType(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&) const @ 0x1146728d in /usr/bin/clickhouse
13. DB::IFunctionOverloadResolver::build(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&) const @ 0x11467ed6 in /usr/bin/clickhouse
14. DB::ActionsDAG::addFunction(std::__1::shared_ptr<DB::IFunctionOverloadResolver> const&, std::__1::vector<DB::ActionsDAG::Node const*, std::__1::allocator<DB::ActionsDAG::Node const*> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) @ 0x11b57c11 in /usr/bin/clickhouse
15. DB::ScopeStack::addFunction(std::__1::shared_ptr<DB::IFunctionOverloadResolver> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) @ 0x1203cd8d in /usr/bin/clickhouse
16. ? @ 0x1204788f in /usr/bin/clickhouse
17. DB::ActionsMatcher::visit(DB::ASTFunction const&, std::__1::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x12042743 in /usr/bin/clickhouse
18. DB::ActionsMatcher::visit(DB::ASTExpressionList&, std::__1::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x12045638 in /usr/bin/clickhouse
19. ? @ 0x1200549c in /usr/bin/clickhouse
20. DB::ExpressionAnalyzer::getRootActions(std::__1::shared_ptr<DB::IAST> const&, bool, std::__1::shared_ptr<DB::ActionsDAG>&, bool) @ 0x1200523e in /usr/bin/clickhouse
21. DB::SelectQueryExpressionAnalyzer::appendSelect(DB::ExpressionActionsChain&, bool) @ 0x12014825 in /usr/bin/clickhouse
22. DB::ExpressionAnalysisResult::ExpressionAnalysisResult(DB::SelectQueryExpressionAnalyzer&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, bool, bool, bool, std::__1::shared_ptr<DB::FilterDAGInfo> const&, std::__1::shared_ptr<DB::FilterDAGInfo> const&, DB::Block const&) @ 0x12019b41 in /usr/bin/clickhouse
23. DB::InterpreterSelectQuery::getSampleBlockImpl() @ 0x125d55a3 in /usr/bin/clickhouse
24. ? @ 0x125cdcf5 in /usr/bin/clickhouse
25. DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context> const&, std::__1::optional<DB::Pipe>, std::__1::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::PreparedSets>) @ 0x125c8a40 in /usr/bin/clickhouse
26. DB::InterpreterSelectWithUnionQuery::buildCurrentChildInterpreter(std::__1::shared_ptr<DB::IAST> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0x126626c2 in /usr/bin/clickhouse
27. DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0x12660440 in /usr/bin/clickhouse
28. DB::InterpreterFactory::get(std::__1::shared_ptr<DB::IAST>&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) @ 0x12587afe in /usr/bin/clickhouse
29. ? @ 0x1296a3a0 in /usr/bin/clickhouse
30. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>, std::__1::optional<DB::FormatSettings> const&) @ 0x1296f98e in /usr/bin/clickhouse
31. DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x134ca797 in /usr/bin/clickhouse
 (version 22.10.3.27 (official build))
2022.11.25 12:21:39.087147 [ 336 ] {} <Warning> StorageKafka (debezium_mysql_log_mailru_score_kafka): Can't get assignment. Will keep trying.
2022.11.25 12:21:40.569294 [ 295 ] {} <Warning> StorageKafka (debezium_mysql_log_nbch_ddsa_kafka): Can't get assignment. Will keep trying.
2022.11.25 12:21:43.026557 [ 292 ] {} <Warning> StorageKafka (debezium_mysql_log_nspk_score_kafka): Can't get assignment. Will keep trying.
2022.11.25 12:21:44.131088 [ 1093 ] {3d111fc7-1d29-41cb-88dd-52ba4071ba16} <Error> executeQuery: Code: 410. DB::Exception: BridgeHelper: clickhouse-library-bridge is not responding: While processing catboostEvaluate('/etc/clickhouse-server/models/catboost_model_1b9ab50eedbe41a5be64a3a224eda68c.cbm', if((toFloat64(cred_sum_overdue_75p_all) IS NULL) OR isInfinite(toFloat64(cred_sum_overdue_75p_all)), nan, toFloat64(cred_sum_overdue_75p_all)), if((toFloat64(no_deep_dpd_without_card_1y_rate) IS NULL) OR isInfinite(toFloat64(no_deep_dpd_without_card_1y_rate)), nan, toFloat64(no_deep_dpd_without_card_1y_rate)), if((toFloat64(amount_outstanding_min_other) IS NULL) OR isInfinite(toFloat64(amount_outstanding_min_other)), nan, toFloat64(amount_outstanding_min_other)), if((toFloat64(es_potential_monthly_instalments) IS NULL) OR isInfinite(toFloat64(es_potential_monthly_instalments)), nan, toFloat64(es_potential_monthly_instalments)), if((toFloat64(amount_outstanding_count_other) IS NULL) OR isInfinite(toFloat64(amount_outstanding_count_other)), nan, toFloat64(amount_outstanding_count_other)), if((toFloat64(bor_data_sex) IS NULL) OR isInfinite(toFloat64(bor_data_sex)), nan, toFloat64(bor_data_sex)), if((toFloat64(credit_life_2_term_without_card_3m_75p) IS NULL) OR isInfinite(toFloat64(credit_life_2_term_without_card_3m_75p)), nan, toFloat64(credit_life_2_term_without_card_3m_75p)), if((toFloat64(credit_life_2_term_without_card_3m_95p) IS NULL) OR isInfinite(toFloat64(credit_life_2_term_without_card_3m_95p)), nan, toFloat64(credit_life_2_term_without_card_3m_95p)), if((toFloat64(credit_life_2_term_75p) IS NULL) OR isInfinite(toFloat64(credit_life_2_term_75p)), nan, toFloat64(credit_life_2_term_75p)), if((toFloat64(amount_outstanding_75p_all) IS NULL) OR isInfinite(toFloat64(amount_outstanding_75p_all)), nan, toFloat64(amount_outstanding_75p_all)), if((toFloat64(credit_without_card_3m_1y_rate) IS NULL) OR isInfinite(toFloat64(credit_without_card_3m_1y_rate)), nan, toFloat64(credit_without_card_3m_1y_rate)), if((toFloat64(days_last_close_date_pdl_without_card_min) IS NULL) OR isInfinite(toFloat64(days_last_close_date_pdl_without_card_min)), nan, toFloat64(days_last_close_date_pdl_without_card_min)), if((toFloat64(bor_pti) IS NULL) OR isInfinite(toFloat64(bor_pti)), nan, toFloat64(bor_pti)), if((toFloat64(nbch_credit_count_rate) IS NULL) OR isInfinite(toFloat64(nbch_credit_count_rate)), nan, toFloat64(nbch_credit_count_rate)), if((toFloat64(current_days_past_75p_max) IS NULL) OR isInfinite(toFloat64(current_days_past_75p_max)), nan, toFloat64(current_days_past_75p_max)), if((toFloat64(closed_pdl_cred_sum_without_card_sum) IS NULL) OR isInfinite(toFloat64(closed_pdl_cred_sum_without_card_sum)), nan, toFloat64(closed_pdl_cred_sum_without_card_sum)), if((toFloat64(max_days_past_due_without_card_6m_95p) IS NULL) OR isInfinite(toFloat64(max_days_past_due_without_card_6m_95p)), nan, toFloat64(max_days_past_due_without_card_6m_95p)), if((toFloat64(active_overdue_2_sum_without_card_max) IS NULL) OR isInfinite(toFloat64(active_overdue_2_sum_without_card_max)), nan, toFloat64(active_overdue_2_sum_without_card_max)), if((toFloat64(es_current_30_ind) IS NULL) OR isInfinite(toFloat64(es_current_30_ind)), nan, toFloat64(es_current_30_ind)), if((toFloat64(credit_life_2_term_il_without_card_1y_avg) IS NULL) OR isInfinite(toFloat64(credit_life_2_term_il_without_card_1y_avg)), nan, toFloat64(credit_life_2_term_il_without_card_1y_avg)), if((toFloat64(credit_life_2_term_pdl_without_card_6m_avg) IS NULL) OR isInfinite(toFloat64(credit_life_2_term_pdl_without_card_6m_avg)), nan, toFloat64(credit_life_2_term_pdl_without_card_6m_avg)), if((toFloat64(from_open_2_open_avg_all) IS NULL) OR isInfinite(toFloat64(from_open_2_open_avg_all)), nan, toFloat64(from_open_2_open_avg_all)), if((toFloat64(es_interest3to12) IS NULL) OR isInfinite(toFloat64(es_interest3to12)), nan, toFloat64(es_interest3to12)), if((toFloat64(credit_sum_all) IS NULL) OR isInfinite(toFloat64(credit_sum_all)), nan, toFloat64(credit_sum_all)), if((toFloat64(nodpd_without_card_6m_rate) IS NULL) OR isInfinite(toFloat64(nodpd_without_card_6m_rate)), nan, toFloat64(nodpd_without_card_6m_rate)), if((toFloat64(es_interest1to12) IS NULL) OR isInfinite(toFloat64(es_interest1to12)), nan, toFloat64(es_interest1to12)), if((toFloat64(credit_life_sum) IS NULL) OR isInfinite(toFloat64(credit_life_sum)), nan, toFloat64(credit_life_sum)), if((toFloat64(bor_data_payments_loans) IS NULL) OR isInfinite(toFloat64(bor_data_payments_loans)), nan, toFloat64(bor_data_payments_loans)), if((toFloat64(max_days_past_due_without_card_1y_max) IS NULL) OR isInfinite(toFloat64(max_days_past_due_without_card_1y_max)), nan, toFloat64(max_days_past_due_without_card_1y_max)), ifNull(bor_data_post, '__MISSING__')) AS prediction, 1. / (1 + exp(-prediction)) AS probability, round(probability * 1000) AS score, credit_id, cred_sum_overdue_75p_all, no_deep_dpd_without_card_1y_rate, amount_outstanding_min_other, es_potential_monthly_instalments, amount_outstanding_count_other, bor_data_sex, credit_life_2_term_without_card_3m_75p, credit_life_2_term_without_card_3m_95p, credit_life_2_term_75p, bor_data_post, amount_outstanding_75p_all, credit_without_card_3m_1y_rate, days_last_close_date_pdl_without_card_min, bor_pti, nbch_credit_count_rate, current_days_past_75p_max, closed_pdl_cred_sum_without_card_sum, max_days_past_due_without_card_6m_95p, active_overdue_2_sum_without_card_max, es_current_30_ind, credit_life_2_term_il_without_card_1y_avg, credit_life_2_term_pdl_without_card_6m_avg, from_open_2_open_avg_all, es_interest3to12, credit_sum_all, nodpd_without_card_6m_rate, es_interest1to12, credit_life_sum, bor_data_payments_loans, max_days_past_due_without_card_1y_max. (EXTERNAL_SERVER_IS_NOT_RESPONDING) (version 22.10.3.27 (official build)) (from 93.125.2.75:60648) (in query: SELECT catboostEvaluate('/etc/clickhouse-server/models/catboost_model_1b9ab50eedbe41a5be64a3a224eda68c.cbm', if(isNull(toFloat64(cred_sum_overdue_75p_all)) OR isInfinite(toFloat64(cred_sum_overdue_75p_all)), nan, toFloat64(cred_sum_overdue_75p_all)), if(isNull(toFloat64(no_deep_dpd_without_card_1y_rate)) OR isInfinite(toFloat64(no_deep_dpd_without_card_1y_rate)), nan, toFloat64(no_deep_dpd_without_card_1y_rate)), if(isNull(toFloat64(amount_outstanding_min_other)) OR isInfinite(toFloat64(amount_outstanding_min_other)), nan, toFloat64(amount_outstanding_min_other)), if(isNull(toFloat64(es_potential_monthly_instalments)) OR isInfinite(toFloat64(es_potential_monthly_instalments)), nan, toFloat64(es_potential_monthly_instalments)), if(isNull(toFloat64(amount_outstanding_count_other)) OR isInfinite(toFloat64(amount_outstanding_count_other)), nan, toFloat64(amount_outstanding_count_other)), if(isNull(toFloat64(bor_data_sex)) OR isInfinite(toFloat64(bor_data_sex)), nan, toFloat64(bor_data_sex)), if(isNull(toFloat64(credit_life_2_term_without_card_3m_75p)) OR isInfinite(toFloat64(credit_life_2_term_without_card_3m_75p)), nan, toFloat64(credit_life_2_term_without_card_3m_75p)), if(isNull(toFloat64(credit_life_2_term_without_card_3m_95p)) OR isInfinite(toFloat64(credit_life_2_term_without_card_3m_95p)), nan, toFloat64(credit_life_2_term_without_card_3m_95p)), if(isNull(toFloat64(credit_life_2_term_75p)) OR isInfinite(toFloat64(credit_life_2_term_75p)), nan, toFloat64(credit_life_2_term_75p)), if(isNull(toFloat64(amount_outstanding_75p_all)) OR isInfinite(toFloat64(amount_outstanding_75p_all)), nan, toFloat64(amount_outstanding_75p_all)), if(isNull(toFloat64(credit_without_card_3m_1y_rate)) OR isInfinite(toFloat64(credit_without_card_3m_1y_rate)), nan, toFloat64(credit_without_card_3m_1y_rate)), if(isNull(toFloat64(days_last_close_date_pdl_without_card_min)) OR isInfinite(toFloat64(days_last_close_date_pdl_without_card_min)), nan, toFloat64(days_last_close_date_pdl_without_card_min)), if(isNull(toFloat64(bor_pti)) OR isInfinite(toFloat64(bor_pti)), nan, toFloat64(bor_pti)), if(isNull(toFloat64(nbch_credit_count_rate)) OR isInfinite(toFloat64(nbch_credit_count_rate)), nan, toFloat64(nbch_credit_count_rate)), if(isNull(toFloat64(current_days_past_75p_max)) OR isInfinite(toFloat64(current_days_past_75p_max)), nan, toFloat64(current_days_past_75p_max)), if(isNull(toFloat64(closed_pdl_cred_sum_without_card_sum)) OR isInfinite(toFloat64(closed_pdl_cred_sum_without_card_sum)), nan, toFloat64(closed_pdl_cred_sum_without_card_sum)), if(isNull(toFloat64(max_days_past_due_without_card_6m_95p)) OR isInfinite(toFloat64(max_days_past_due_without_card_6m_95p)), nan, toFloat64(max_days_past_due_without_card_6m_95p)), if(isNull(toFloat64(active_overdue_2_sum_without_card_max)) OR isInfinite(toFloat64(active_overdue_2_sum_without_card_max)), nan, toFloat64(active_overdue_2_sum_without_card_max)), if(isNull(toFloat64(es_current_30_ind)) OR isInfinite(toFloat64(es_current_30_ind)), nan, toFloat64(es_current_30_ind)), if(isNull(toFloat64(credit_life_2_term_il_without_card_1y_avg)) OR isInfinite(toFloat64(credit_life_2_term_il_without_card_1y_avg)), nan, toFloat64(credit_life_2_term_il_without_card_1y_avg)), if(isNull(toFloat64(credit_life_2_term_pdl_without_card_6m_avg)) OR isInfinite(toFloat64(credit_life_2_term_pdl_without_card_6m_avg)), nan, toFloat64(credit_life_2_term_pdl_without_card_6m_avg)), if(isNull(toFloat64(from_open_2_open_avg_all)) OR isInfinite(toFloat64(from_open_2_open_avg_all)), nan, toFloat64(from_open_2_open_avg_all)), if(isNull(toFloat64(es_interest3to12)) OR isInfinite(toFloat64(es_interest3to12)), nan, toFloat64(es_interest3to12)), if(isNull(toFloat64(credit_sum_all)) OR isInfinite(toFloat64(credit_sum_all)), nan, toFloat64(credit_sum_all)), if(isNull(toFloat64(nodpd_without_card_6m_rate)) OR isInfinite(toFloat64(nodpd_without_card_6m_rate)), nan, toFloat64(nodpd_without_card_6m_rate)), if(isNull(toFloat64(es_interest1to12)) OR isInfinite(toFloat64(es_interest1to12)), nan, toFloat64(es_interest1to12)), if(isNull(toFloat64(credit_life_sum)) OR isInfinite(toFloat64(credit_life_sum)), nan, toFloat64(credit_life_sum)), if(isNull(toFloat64(bor_data_payments_loans)) OR isInfinite(toFloat64(bor_data_payments_loans)), nan, toFloat64(bor_data_payments_loans)), if(isNull(toFloat64(max_days_past_due_without_card_1y_max)) OR isInfinite(toFloat64(max_days_past_due_without_card_1y_max)), nan, toFloat64(max_days_past_due_without_card_1y_max)), ifNull(bor_data_post, '__MISSING__')) AS prediction, 1. / (1 + exp(-prediction)) AS probability, round(probability * 1000) AS score, credit_id, cred_sum_overdue_75p_all, no_deep_dpd_without_card_1y_rate, amount_outstanding_min_other, es_potential_monthly_instalments, amount_outstanding_count_other, bor_data_sex, credit_life_2_term_without_card_3m_75p, credit_life_2_term_without_card_3m_95p, credit_life_2_term_75p, bor_data_post, amount_outstanding_75p_all, credit_without_card_3m_1y_rate, days_last_close_date_pdl_without_card_min, bor_pti, nbch_credit_count_rate, current_days_past_75p_max, closed_pdl_cred_sum_without_card_sum, max_days_past_due_without_card_6m_95p, active_overdue_2_sum_without_card_max, es_current_30_ind, credit_life_2_term_il_without_card_1y_avg, credit_life_2_term_pdl_without_card_6m_avg, from_open_2_open_avg_all, es_interest3to12, credit_sum_all, nodpd_without_card_6m_rate, es_interest1to12, credit_life_sum, bor_data_payments_loans, max_days_past_due_without_card_1y_max FROM ru_master.scoring_collect_aggregates_by_credit sca final cross join (select * from ru_master.bch_credit_aggregation_view_by_external_ids settings custom_equifax_external_id = 12943210 , custom_nbch_external_id = 13427929 , custom_ucb_external_id = 13689756) bch where credit_id = 19633684), Stack trace (when copying this message, always include the lines below):

0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0xce3a81a in /usr/bin/clickhouse
1. DB::IBridgeHelper::startBridgeSync() @ 0x1346ba29 in /usr/bin/clickhouse
2. DB::CatBoostLibraryBridgeHelper::getTreeCount() @ 0x134639b6 in /usr/bin/clickhouse
3. ? @ 0x9ab58c7 in /usr/bin/clickhouse
4. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&) const @ 0x114675ea in /usr/bin/clickhouse
5. DB::IFunctionOverloadResolver::getReturnType(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&) const @ 0x1146728d in /usr/bin/clickhouse
6. DB::IFunctionOverloadResolver::build(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&) const @ 0x11467ed6 in /usr/bin/clickhouse
7. DB::ActionsDAG::addFunction(std::__1::shared_ptr<DB::IFunctionOverloadResolver> const&, std::__1::vector<DB::ActionsDAG::Node const*, std::__1::allocator<DB::ActionsDAG::Node const*> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) @ 0x11b57c11 in /usr/bin/clickhouse
8. DB::ScopeStack::addFunction(std::__1::shared_ptr<DB::IFunctionOverloadResolver> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) @ 0x1203cd8d in /usr/bin/clickhouse
9. ? @ 0x1204788f in /usr/bin/clickhouse
10. DB::ActionsMatcher::visit(DB::ASTFunction const&, std::__1::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x12042743 in /usr/bin/clickhouse
11. DB::ActionsMatcher::visit(DB::ASTExpressionList&, std::__1::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x12045638 in /usr/bin/clickhouse
12. ? @ 0x1200549c in /usr/bin/clickhouse
13. DB::ExpressionAnalyzer::getRootActions(std::__1::shared_ptr<DB::IAST> const&, bool, std::__1::shared_ptr<DB::ActionsDAG>&, bool) @ 0x1200523e in /usr/bin/clickhouse
14. DB::SelectQueryExpressionAnalyzer::appendSelect(DB::ExpressionActionsChain&, bool) @ 0x12014825 in /usr/bin/clickhouse
15. DB::ExpressionAnalysisResult::ExpressionAnalysisResult(DB::SelectQueryExpressionAnalyzer&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, bool, bool, bool, std::__1::shared_ptr<DB::FilterDAGInfo> const&, std::__1::shared_ptr<DB::FilterDAGInfo> const&, DB::Block const&) @ 0x12019b41 in /usr/bin/clickhouse
16. DB::InterpreterSelectQuery::getSampleBlockImpl() @ 0x125d55a3 in /usr/bin/clickhouse
17. ? @ 0x125cdcf5 in /usr/bin/clickhouse
18. DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context> const&, std::__1::optional<DB::Pipe>, std::__1::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::PreparedSets>) @ 0x125c8a40 in /usr/bin/clickhouse
19. DB::InterpreterSelectWithUnionQuery::buildCurrentChildInterpreter(std::__1::shared_ptr<DB::IAST> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0x126626c2 in /usr/bin/clickhouse
20. DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0x12660440 in /usr/bin/clickhouse
21. DB::InterpreterFactory::get(std::__1::shared_ptr<DB::IAST>&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) @ 0x12587afe in /usr/bin/clickhouse
22. ? @ 0x1296a3a0 in /usr/bin/clickhouse
23. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>, std::__1::optional<DB::FormatSettings> const&) @ 0x1296f98e in /usr/bin/clickhouse
24. DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x134ca797 in /usr/bin/clickhouse
25. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x134cebb0 in /usr/bin/clickhouse
26. DB::HTTPServerConnection::run() @ 0x1354c1aa in /usr/bin/clickhouse
27. Poco::Net::TCPServerConnection::start() @ 0x1635308f in /usr/bin/clickhouse
28. Poco::Net::TCPServerDispatcher::run() @ 0x1635541b in /usr/bin/clickhouse
29. Poco::PooledThread::run() @ 0x16510cb2 in /usr/bin/clickhouse
30. Poco::ThreadImpl::runnableEntry(void*) @ 0x1650e43d in /usr/bin/clickhouse
31. ? @ 0x7f9f15026609 in ?

2022.11.25 12:21:44.132340 [ 1093 ] {3d111fc7-1d29-41cb-88dd-52ba4071ba16} <Error> DynamicQueryHandler: Code: 410. DB::Exception: BridgeHelper: clickhouse-library-bridge is not responding: While processing catboostEvaluate('/etc/clickhouse-server/models/catboost_model_1b9ab50eedbe41a5be64a3a224eda68c.cbm', if((toFloat64(cred_sum_overdue_75p_all) IS NULL) OR isInfinite(toFloat64(cred_sum_overdue_75p_all)), nan, toFloat64(cred_sum_overdue_75p_all)), if((toFloat64(no_deep_dpd_without_card_1y_rate) IS NULL) OR isInfinite(toFloat64(no_deep_dpd_without_card_1y_rate)), nan, toFloat64(no_deep_dpd_without_card_1y_rate)), if((toFloat64(amount_outstanding_min_other) IS NULL) OR isInfinite(toFloat64(amount_outstanding_min_other)), nan, toFloat64(amount_outstanding_min_other)), if((toFloat64(es_potential_monthly_instalments) IS NULL) OR isInfinite(toFloat64(es_potential_monthly_instalments)), nan, toFloat64(es_potential_monthly_instalments)), if((toFloat64(amount_outstanding_count_other) IS NULL) OR isInfinite(toFloat64(amount_outstanding_count_other)), nan, toFloat64(amount_outstanding_count_other)), if((toFloat64(bor_data_sex) IS NULL) OR isInfinite(toFloat64(bor_data_sex)), nan, toFloat64(bor_data_sex)), if((toFloat64(credit_life_2_term_without_card_3m_75p) IS NULL) OR isInfinite(toFloat64(credit_life_2_term_without_card_3m_75p)), nan, toFloat64(credit_life_2_term_without_card_3m_75p)), if((toFloat64(credit_life_2_term_without_card_3m_95p) IS NULL) OR isInfinite(toFloat64(credit_life_2_term_without_card_3m_95p)), nan, toFloat64(credit_life_2_term_without_card_3m_95p)), if((toFloat64(credit_life_2_term_75p) IS NULL) OR isInfinite(toFloat64(credit_life_2_term_75p)), nan, toFloat64(credit_life_2_term_75p)), if((toFloat64(amount_outstanding_75p_all) IS NULL) OR isInfinite(toFloat64(amount_outstanding_75p_all)), nan, toFloat64(amount_outstanding_75p_all)), if((toFloat64(credit_without_card_3m_1y_rate) IS NULL) OR isInfinite(toFloat64(credit_without_card_3m_1y_rate)), nan, toFloat64(credit_without_card_3m_1y_rate)), if((toFloat64(days_last_close_date_pdl_without_card_min) IS NULL) OR isInfinite(toFloat64(days_last_close_date_pdl_without_card_min)), nan, toFloat64(days_last_close_date_pdl_without_card_min)), if((toFloat64(bor_pti) IS NULL) OR isInfinite(toFloat64(bor_pti)), nan, toFloat64(bor_pti)), if((toFloat64(nbch_credit_count_rate) IS NULL) OR isInfinite(toFloat64(nbch_credit_count_rate)), nan, toFloat64(nbch_credit_count_rate)), if((toFloat64(current_days_past_75p_max) IS NULL) OR isInfinite(toFloat64(current_days_past_75p_max)), nan, toFloat64(current_days_past_75p_max)), if((toFloat64(closed_pdl_cred_sum_without_card_sum) IS NULL) OR isInfinite(toFloat64(closed_pdl_cred_sum_without_card_sum)), nan, toFloat64(closed_pdl_cred_sum_without_card_sum)), if((toFloat64(max_days_past_due_without_card_6m_95p) IS NULL) OR isInfinite(toFloat64(max_days_past_due_without_card_6m_95p)), nan, toFloat64(max_days_past_due_without_card_6m_95p)), if((toFloat64(active_overdue_2_sum_without_card_max) IS NULL) OR isInfinite(toFloat64(active_overdue_2_sum_without_card_max)), nan, toFloat64(active_overdue_2_sum_without_card_max)), if((toFloat64(es_current_30_ind) IS NULL) OR isInfinite(toFloat64(es_current_30_ind)), nan, toFloat64(es_current_30_ind)), if((toFloat64(credit_life_2_term_il_without_card_1y_avg) IS NULL) OR isInfinite(toFloat64(credit_life_2_term_il_without_card_1y_avg)), nan, toFloat64(credit_life_2_term_il_without_card_1y_avg)), if((toFloat64(credit_life_2_term_pdl_without_card_6m_avg) IS NULL) OR isInfinite(toFloat64(credit_life_2_term_pdl_without_card_6m_avg)), nan, toFloat64(credit_life_2_term_pdl_without_card_6m_avg)), if((toFloat64(from_open_2_open_avg_all) IS NULL) OR isInfinite(toFloat64(from_open_2_open_avg_all)), nan, toFloat64(from_open_2_open_avg_all)), if((toFloat64(es_interest3to12) IS NULL) OR isInfinite(toFloat64(es_interest3to12)), nan, toFloat64(es_interest3to12)), if((toFloat64(credit_sum_all) IS NULL) OR isInfinite(toFloat64(credit_sum_all)), nan, toFloat64(credit_sum_all)), if((toFloat64(nodpd_without_card_6m_rate) IS NULL) OR isInfinite(toFloat64(nodpd_without_card_6m_rate)), nan, toFloat64(nodpd_without_card_6m_rate)), if((toFloat64(es_interest1to12) IS NULL) OR isInfinite(toFloat64(es_interest1to12)), nan, toFloat64(es_interest1to12)), if((toFloat64(credit_life_sum) IS NULL) OR isInfinite(toFloat64(credit_life_sum)), nan, toFloat64(credit_life_sum)), if((toFloat64(bor_data_payments_loans) IS NULL) OR isInfinite(toFloat64(bor_data_payments_loans)), nan, toFloat64(bor_data_payments_loans)), if((toFloat64(max_days_past_due_without_card_1y_max) IS NULL) OR isInfinite(toFloat64(max_days_past_due_without_card_1y_max)), nan, toFloat64(max_days_past_due_without_card_1y_max)), ifNull(bor_data_post, '__MISSING__')) AS prediction, 1. / (1 + exp(-prediction)) AS probability, round(probability * 1000) AS score, credit_id, cred_sum_overdue_75p_all, no_deep_dpd_without_card_1y_rate, amount_outstanding_min_other, es_potential_monthly_instalments, amount_outstanding_count_other, bor_data_sex, credit_life_2_term_without_card_3m_75p, credit_life_2_term_without_card_3m_95p, credit_life_2_term_75p, bor_data_post, amount_outstanding_75p_all, credit_without_card_3m_1y_rate, days_last_close_date_pdl_without_card_min, bor_pti, nbch_credit_count_rate, current_days_past_75p_max, closed_pdl_cred_sum_without_card_sum, max_days_past_due_without_card_6m_95p, active_overdue_2_sum_without_card_max, es_current_30_ind, credit_life_2_term_il_without_card_1y_avg, credit_life_2_term_pdl_without_card_6m_avg, from_open_2_open_avg_all, es_interest3to12, credit_sum_all, nodpd_without_card_6m_rate, es_interest1to12, credit_life_sum, bor_data_payments_loans, max_days_past_due_without_card_1y_max. (EXTERNAL_SERVER_IS_NOT_RESPONDING), Stack trace (when copying this message, always include the lines below):

0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0xce3a81a in /usr/bin/clickhouse
1. DB::IBridgeHelper::startBridgeSync() @ 0x1346ba29 in /usr/bin/clickhouse
2. DB::CatBoostLibraryBridgeHelper::getTreeCount() @ 0x134639b6 in /usr/bin/clickhouse
3. ? @ 0x9ab58c7 in /usr/bin/clickhouse
4. DB::IFunctionOverloadResolver::getReturnTypeWithoutLowCardinality(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&) const @ 0x114675ea in /usr/bin/clickhouse
5. DB::IFunctionOverloadResolver::getReturnType(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&) const @ 0x1146728d in /usr/bin/clickhouse
6. DB::IFunctionOverloadResolver::build(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&) const @ 0x11467ed6 in /usr/bin/clickhouse
7. DB::ActionsDAG::addFunction(std::__1::shared_ptr<DB::IFunctionOverloadResolver> const&, std::__1::vector<DB::ActionsDAG::Node const*, std::__1::allocator<DB::ActionsDAG::Node const*> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) @ 0x11b57c11 in /usr/bin/clickhouse
8. DB::ScopeStack::addFunction(std::__1::shared_ptr<DB::IFunctionOverloadResolver> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) @ 0x1203cd8d in /usr/bin/clickhouse
9. ? @ 0x1204788f in /usr/bin/clickhouse
10. DB::ActionsMatcher::visit(DB::ASTFunction const&, std::__1::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x12042743 in /usr/bin/clickhouse
11. DB::ActionsMatcher::visit(DB::ASTExpressionList&, std::__1::shared_ptr<DB::IAST> const&, DB::ActionsMatcher::Data&) @ 0x12045638 in /usr/bin/clickhouse
12. ? @ 0x1200549c in /usr/bin/clickhouse
13. DB::ExpressionAnalyzer::getRootActions(std::__1::shared_ptr<DB::IAST> const&, bool, std::__1::shared_ptr<DB::ActionsDAG>&, bool) @ 0x1200523e in /usr/bin/clickhouse
14. DB::SelectQueryExpressionAnalyzer::appendSelect(DB::ExpressionActionsChain&, bool) @ 0x12014825 in /usr/bin/clickhouse
15. DB::ExpressionAnalysisResult::ExpressionAnalysisResult(DB::SelectQueryExpressionAnalyzer&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, bool, bool, bool, std::__1::shared_ptr<DB::FilterDAGInfo> const&, std::__1::shared_ptr<DB::FilterDAGInfo> const&, DB::Block const&) @ 0x12019b41 in /usr/bin/clickhouse
16. DB::InterpreterSelectQuery::getSampleBlockImpl() @ 0x125d55a3 in /usr/bin/clickhouse
17. ? @ 0x125cdcf5 in /usr/bin/clickhouse
18. DB::InterpreterSelectQuery::InterpreterSelectQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context> const&, std::__1::optional<DB::Pipe>, std::__1::shared_ptr<DB::IStorage> const&, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::shared_ptr<DB::PreparedSets>) @ 0x125c8a40 in /usr/bin/clickhouse
19. DB::InterpreterSelectWithUnionQuery::buildCurrentChildInterpreter(std::__1::shared_ptr<DB::IAST> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0x126626c2 in /usr/bin/clickhouse
20. DB::InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0x12660440 in /usr/bin/clickhouse
21. DB::InterpreterFactory::get(std::__1::shared_ptr<DB::IAST>&, std::__1::shared_ptr<DB::Context>, DB::SelectQueryOptions const&) @ 0x12587afe in /usr/bin/clickhouse
22. ? @ 0x1296a3a0 in /usr/bin/clickhouse
23. DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, bool, std::__1::shared_ptr<DB::Context>, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>, std::__1::optional<DB::FormatSettings> const&) @ 0x1296f98e in /usr/bin/clickhouse
24. DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x134ca797 in /usr/bin/clickhouse
25. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x134cebb0 in /usr/bin/clickhouse
26. DB::HTTPServerConnection::run() @ 0x1354c1aa in /usr/bin/clickhouse
27. Poco::Net::TCPServerConnection::start() @ 0x1635308f in /usr/bin/clickhouse
28. Poco::Net::TCPServerDispatcher::run() @ 0x1635541b in /usr/bin/clickhouse
29. Poco::PooledThread::run() @ 0x16510cb2 in /usr/bin/clickhouse
30. Poco::ThreadImpl::runnableEntry(void*) @ 0x1650e43d in /usr/bin/clickhouse
31. ? @ 0x7f9f15026609 in ?
 (version 22.10.3.27 (official build))

CH processes (after query):

(base) root@glprod-clickhouse:~# docker exec -it root_clickhouse-server-mmru_1 bash
root@16f39704387b:/# ps aux | grep click
clickho+     1 73.6  1.0 13514240 1394660 ?    Ssl  12:19   1:04 /usr/bin/clickhouse-server --config-file=/etc/clickhouse-server/config.xml
root      1092  0.0  0.0   6304   716 pts/0    S+   12:21   0:00 grep --color=auto click
root@16f39704387b:/# ps aux | grep click
clickho+     1 73.1  1.1 13633032 1438400 ?    Ssl  12:19   1:24 /usr/bin/clickhouse-server --config-file=/etc/clickhouse-server/config.xml
clickho+  1095  0.0  0.0      0     0 ?        Z    12:21   0:00 [clickhouse-libr] <defunct>
root      1098  0.0  0.0   6304   656 pts/0    S+   12:21   0:00 grep --color=auto click