kafka-ui: Can't connect to azure event hub kafka interface

Describe the bug

We wanted to monitor our azure event hub, which we mainly use as a kafka server. The kafka interface works and our backend services can communicate with the kafka interface

Monitoring also works with kafka-tool/offset-explorer

But kafka-ui fails to connect to the hub with this error message:

2022-08-12 14:27:25,701 ERROR [parallel-6] c.p.k.u.s.MetricsService: Failed to collect cluster prod-event-hub info
java.lang.IllegalStateException: Error while creating AdminClient for Cluster prod-event-hub
        at com.provectus.kafka.ui.service.AdminClientServiceImpl.lambda$createAdminClient$3(AdminClientServiceImpl.java:45)
        at reactor.core.publisher.Mono.lambda$onErrorMap$31(Mono.java:3733)
        at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:94)
        at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onError(FluxMapFuseable.java:140)
        at reactor.core.publisher.MonoFlatMap$FlatMapMain.secondError(MonoFlatMap.java:192)
        at reactor.core.publisher.MonoFlatMap$FlatMapInner.onError(MonoFlatMap.java:259)
        at reactor.core.publisher.MonoPublishOn$PublishOnSubscriber.run(MonoPublishOn.java:187)
        at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68)
        at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:830)
Caused by: org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition.

The kafka UI has full admin rights on the assigned user

Set up

Kafka-UI is setup with a standard docker compose:

version: '2.7'

services:
  kafka-ui:
    image: provectuslabs/kafka-ui
    container_name: kafka-ui
    ports:
      - "8085:8080"
    restart: always
    environment:
      - KAFKA_CLUSTERS_0_NAME=prod-event-hub
      - KAFKA_CLUSTERS_0_PROPERTIES_SECURITY_PROTOCOL=SASL_SSL
      - KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=***.servicebus.windows.net:9093
      - KAFKA_CLUSTERS_0_PROPERTIES_SASL_MECHANISM=PLAIN
      - KAFKA_CLUSTERS_0_PROPERTIES_SASL_JAAS_CONFIG=org.apache.kafka.common.security.plain.PlainLoginModule required username='$$ConnectionString' password='***;

Expected behavior

A functioning connection with the azure event hub

About this issue

Original URL
State: closed
Created 2 years ago
Reactions: 5
Comments: 22 (9 by maintainers)

Most upvoted comments

Hi,

Im interested on the connection to Azure Events Hub too.

Thanks!!

srferron on Aug 22, 2022

Can confirm that it works with the newest master branch image

WizzardMaker on Mar 24, 2023

Worked like a charm for me. Great job, Thanks!

mvitor on Mar 31, 2023

@iliax @WizzardMaker After verifying, my kafka ui still does not connect eventhub. Could you give connect example of docker-compose.yaml ? My error message is below:

kafka-ui  | 2023-03-27 01:02:12,188 WARN  [kafka-admin-client-thread | kafka-ui-admin-1679878922-2] o.a.k.c.NetworkClient: [AdminClient clientId=kafka-ui-admin-1679878922-2] Connection to node -1 (<my-eventhub-namespace>.servicebus.windows.net/40.78.253.136:9093) could not be established. Broker may not be available.

I also provide my docker-compose.yaml for your reference:

version: '3'

services:
  kafka-ui:
    image: provectuslabs/kafka-ui:master
    container_name: kafka-ui
    ports:
      - "8080:8080"
    restart: always
    environment:
      - KAFKA_CLUSTERS_0_NAME=prod-event-hub
      - KAFKA_CLUSTERS_0_PROPERTIES_SECURITY_PROTOCOL=SASL_SSL
      - KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=<my-eventhub-namespaces>.servicebus.windows.net:9093
      - KAFKA_CLUSTERS_0_PROPERTIES_SASL_MECHANISM=PLAIN
      - KAFKA_CLUSTERS_0_PROPERTIES_SASL_JAAS_CONFIG=org.apache.kafka.common.security.plain.PlainLoginModule required username='$ConnectionString' password='Endpoint=sb://<my-eventhub-namespaces>.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=<my-key>';

That error message looks more like a general connection problem, and not the problem discussed in this issue.

Are you sure, that your at all able to connect to your event hub at that address? (What is <my-eventhub-namespaces>? Make sure that this is the name of the Azure resource - Check your “Host name” Property of the Event Hub Resource)

WizzardMaker on Mar 27, 2023

@FireDrunk fortunately, we’re already working on this 😃

Haarolean on Mar 22, 2023

Thanks for the feedback @Haarolean !

srferron on Aug 24, 2022