confluent-kafka-dotnet: QueryWatermarkOffsets throws "Broker: Leader not available" if proceeding to fast

The following has been observed using Confluent.Kafka 0.9.4-preview8 agains Kafka_2.12-0.10.2.0 on Windows 10 using the following code:

var config = new Dictionary<string, object>()
{
   { "enable.auto.commit", false },
   { "group.id" , "Foo" },
   { "bootstrap.servers", clusterConnectionString }
};
using (var consumer = new Consumer(config))
{
  // The following line throws, but not if a breakpoint is placed. 
  var mark = consumer.QueryWatermarkOffsets(new TopicPartition("Bar", 0));
}

The line with QueryWatermarkOffsets throws the exception if the code runs uninterrupted. The topic “Bar” exist and has low-high watermarks 0-3. Placing a breakpoint at the line itself solves the problem. Assuming that this was due to an issue with the timing between some async initialization of consumer-metadata and the call to QueryWatermarkOffsets, I tried looping and catching the specific error with a small Thread.Sleep, hoping that it would eventually go away. The problem persisted after 10 1-seconds sleeps.

Perhaps some kind of internal retry or refresh of caches are needed?

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Reactions: 2
  • Comments: 16 (9 by maintainers)

Most upvoted comments

Shouldn’t this rather be labeled as bug instead of enhancement? I’m facing the same issue. Using Confluent.Kafka 0.11 connecting to Kafka 0.9.0.1

I believe this was fixed in v1.7.0: d12c4a106

Any updates on the above issue? I am facing the same exact problem.

Hmm, I must have closed it by mistake. I did not intend to. The issue has been reopened.