vector: SSL handshake failed when sources is kafka
A note for the community
- Please vote on this issue by adding a π reaction to the original issue to help the community and maintainers prioritize this request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
Problem
SSL handshake failed
Configuration
[sources.kafka_log_topics]
type = "kafka"
bootstrap_servers = "x.x.x.x:9093,x.x.x.x:9093,x.x.x.x:9093"
group_id = "nginxLogGroup"
#librdkafka_options.client.id = "vector_node1"
auto_offset_reset = "latest"
topics = ["hftopic"]
decoding.codec = "json"
tls.enabled = true
#tls.alpn_protocols = ["TLS1.2"]
tls.ca_file = "/opt/certs/ca.crt"
tls.crt_file = "/opt/certs/client.crt"
tls.key_file = "/opt/certs/client.key"
#tls.key_pass = "xxxxxxxxxxx"
# tls.verify_hostname = false
#tls.verify_certificate = false
[transforms.reduce_logs]
type = "filter"
inputs = ["kafka_log_topics"]
condition = '''
msg = parse_json!(.message)
starts_with(string!(msg.http_authorization), "xxxxxxxx") && parse_int!(msg.body_bytes_sent) > 0
'''
[transforms.test_logs_transform]
type = "remap"
inputs = ["reduce_logs"]
source = """
. = parse_json!(.message)
del(.bytes_sent)
del(.http_user_agent)
del(.remote_addr)
del(.request)
del(.status)
epoch = parse_int!(split!(.msec, ".")[0])
del(.msec)
ym_str = format_timestamp!(from_unix_timestamp!(epoch), "%Y%m", "Asia/Shanghai")
.ym = parse_int!(ym_str)
.http_authorization = replace(string!(.http_authorization), "tlspxy ", "")
.http_authorization = parse_int!(.http_authorization)
.body_bytes_sent = parse_int!(.body_bytes_sent)
"""
[sinks.file]
inputs = ["test_logs_transform"]
type = "file"
encoding.codec = "json"
path = "/tmp/test.out"
2023-11-16T02:49:24.350699Z ERROR source{component_kind="source" component_id=kafka_log_topics component_type=kafka}:kafka_source: rdkafka::client: librdkafka: Global error: SSL (Local: SSL error): ssl://x.x.x.x:9093/bootstrap: SSL handshake failed: ssl/statem/statem_clnt.c:1890:(unknown function) error:0A000086:SSL routines::certificate verify failed: broker certificate could not be verified, verify that ssl.ca.location is correctly configured or root CA certificates are installed (install ca-certificates package) (after 6ms in state SSL_HANDSHAKE)
2023-11-16T02:49:25.349162Z ERROR source{component_kind="source" component_id=kafka_log_topics component_type=kafka}:kafka_source: rdkafka::client: librdkafka: Global error: SSL (Local: SSL error): ssl://x.x.x.x:9093/bootstrap: SSL handshake failed: ssl/statem/statem_clnt.c:1890:(unknown function) error:0A000086:SSL routines::certificate verify failed: broker certificate could not be verified, verify that ssl.ca.location is correctly configured or root CA certificates are installed (install ca-certificates package) (after 6ms in state SSL_HANDSHAKE)
2023-11-16T02:49:26.347125Z ERROR source{component_kind="source" component_id=kafka_log_topics component_type=kafka}:kafka_source: rdkafka::client: librdkafka: Global error: SSL (Local: SSL error): ssl://x.x.x.x:9093/bootstrap: SSL handshake failed: ssl/statem/statem_clnt.c:1890:(unknown function) error:0A000086:SSL routines::certificate verify failed: broker certificate could not be verified, verify that ssl.ca.location is correctly configured or root CA certificates are installed (install ca-certificates package) (after 4ms in state SSL_HANDSHAKE)
2023-11-16T02:49:26.347168Z ERROR source{component_kind="source" component_id=kafka_log_topics component_type=kafka}:kafka_source: rdkafka::client: librdkafka: Global error: AllBrokersDown (Local: All broker connections are down): 3/3 brokers are down
2023-11-16T02:49:27.348611Z ERROR source{component_kind="source" component_id=kafka_log_topics component_type=kafka}:kafka_source: rdkafka::client: librdkafka: Global error: SSL (Local: SSL error): ssl://x.x.x.x:9093/bootstrap: SSL handshake failed: ssl/statem/statem_clnt.c:1890:(unknown function) error:0A000086:SSL routines::certificate verify failed: broker certificate could not be verified, verify that ssl.ca.location is correctly configured or root CA certificates are installed (install ca-certificates package) (after 5ms in state SSL_HANDSHAKE, 1 identical error(s) suppressed)
2023-11-16T02:49:28.349455Z ERROR source{component_kind="source" component_id=kafka_log_topics component_type=kafka}:kafka_source: rdkafka::client: librdkafka: Global error: SSL (Local: SSL error): ssl://x.x.x.x:9093/bootstrap: SSL handshake failed: ssl/statem/statem_clnt.c:1890:(unknown function) error:0A000086:SSL routines::certificate verify failed: broker certificate could not be verified, verify that ssl.ca.location is correctly configured or root CA certificates are installed (install ca-certificates package) (after 6ms in state SSL_HANDSHAKE, 1 identical error(s) suppressed)
### Version
vector 0.34.0 (x86_64-unknown-linux-gnu c909b66 2023-11-07 15:07:26.748571656)
### Debug Output
_No response_
### Example Data
_No response_
### Additional Context
_No response_
### References
_No response_
About this issue
- Original URL
- State: closed
- Created 8 months ago
- Reactions: 1
- Comments: 17 (7 by maintainers)
Itβs OK when I build vector from master branch source code