eventing: Knative broker Not able to stream to Event Hubs in Azure.

Describe the bug The issue: I installed Knative-eventing with Knative kafka-broker to streaming and consume messages from Azure Event Hubs. However I’m getting Failed to create topic:

$kubectl get broker
NAME         URL   AGE     READY   REASON
lasbroker1         4m14s   False   Failed to create topic: knative-broker-default-lasbroker1

Expected behavior Produce/stream and consume messages/events on Azure Event Hub via the Knative Kafka Broker.

To Reproduce I’m following this guide https://knative.dev/docs/eventing/broker/kafka-broker/.

Install Knative-eventing:

kubectl apply -f https://github.com/knative/eventing/releases/download/v0.26.0/eventing-crds.yaml
kubectl apply -f https://github.com/knative/eventing/releases/download/v0.26.0/eventing-core.yaml

No Strimzi need here because I have my Event Hub working on “my-cluster-fqdn.servicebus.windows.net:9093” Yes, port 9093

Install a default Channel (messaging) layer Here https://knative.dev/docs/admin/install/eventing/install-eventing-with-yaml/ says to use this https://github.com/knative-sandbox/eventing-kafka for this step.

The page says: Note: Replace REPLACE_WITH_CLUSTER_URL in the yaml with the URI to the Kafka Cluster (eg. my-cluster-kafka-bootstrap.kafka:9092/) So I downloaded each yaml file and changed inside to my-cluster-fqdn.servicebus.windows.net:9093

# Install the Kafka Source
kubectl apply -f https://storage.googleapis.com/knative-nightly/eventing-kafka/latest/source.yaml

# Install the Kafka "Consolidated" Channel
kubectl apply -f https://storage.googleapis.com/knative-nightly/eventing-kafka/latest/channel-consolidated.yaml

# Install the Kafka "Distributed" Channel
kubectl apply -f https://storage.googleapis.com/knative-nightly/eventing-kafka/latest/channel-distributed.yaml

Then the next step is create kafka broker:

apiVersion: eventing.knative.dev/v1
kind: Broker
metadata:
  annotations:
    # case-sensitive
    eventing.knative.dev/broker.class: Kafka
  name: lasbroker1
  namespace: default
spec:
  # Configuration specific to this broker.
  config:
    apiVersion: v1
    kind: ConfigMap
    name: kafka-broker-config
    namespace: knative-eventing
    
  # Optional dead letter sink, you can specify either:
  #  - deadLetterSink.ref, which is a reference to a Callable
  #  - deadLetterSink.uri, which is an absolute URI to a Callable (It can potentially be out of the Kubernetes cluster)
  delivery:
    deadLetterSink:
      ref:
        apiVersion: serving.knative.dev/v1
        kind: Service
        name: dlq-service
        data: 
          default.topic.partitions: "10"
          default.topic.replication.factor: "1"
          bootstrap.servers: "my-cluster-fqdn.servicebus.windows.net:9093"

kubectl -f apply my-broker.yaml

After all this, I get the following error and cannot move forward because of that:

soareslo:~$kubectl get broker
NAME         URL   AGE   READY   REASON
lasbroker1         31m   False   Failed to create topic: knative-broker-default-lasbroker1

Knative release version latest

Additional context I really appreciate any support.

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 36 (19 by maintainers)

Most upvoted comments

Thanks, closing this for now.

pierDipi on Nov 2, 2021

The only one difference between yours and mine is that I have used --from-literal=user=“$ConnectionString” (“”) and you have used --from-literal=user=‘$ConnectionString’ (‘’).

Ah, ok, bash interprets $ConnectionString as a variable inside double-quotes.

pierDipi on Nov 2, 2021

broker.yaml

apiVersion: eventing.knative.dev/v1
kind: Broker
metadata:
  annotations:
    # case-sensitive
    eventing.knative.dev/broker.class: Kafka
  name: lasbroker1
  namespace: default
spec:
  # Configuration specific to this broker.
  config:
    apiVersion: v1
    kind: ConfigMap
    name: kafka-broker-config
    namespace: knative-eventing
    
  # Optional dead letter sink, you can specify either:
  #  - deadLetterSink.ref, which is a reference to a Callable
  #  - deadLetterSink.uri, which is an absolute URI to a Callable (It can potentially be out of the Kubernetes cluster)
  delivery:
    deadLetterSink:
      ref:
        apiVersion: serving.knative.dev/v1
        kind: Service
        name: dlq-service

kafka-broker-config.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: kafka-broker-config
  namespace: knative-eventing
data:
  # Number of topic partitions
  default.topic.partitions: "10"
  # Replication factor of topic messages.
  default.topic.replication.factor: "1"
  # A comma separated list of bootstrap servers. (It can be in or out the k8s cluster)
  bootstrap.servers: "my-cluster-here.servicebus.windows.net:9093"
  # Reference a Secret called mysecret
  auth.secret.ref.name: "mysecret"

This is what you want

pierDipi on Oct 29, 2021