confluent-kafka-go: librdkafka cannot be installed on Debian Stretch

Description

libssl conflict results in inability to install librdkafka-dev=1.1.0~1confluent5.3.0-1 on Debian

How to reproduce

# add-apt-repository [...] && apt update && apt install librdkafka-dev

Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 librdkafka-dev : Depends: librdkafka1 (= 1.1.0~1confluent5.3.0-1) but it is not going to be installed
                  Depends: librdkafka++1 (= 1.1.0~1confluent5.3.0-1) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
Exited with code 100

# apt-get install librdkafka-dev=1.1.0~1confluent5.3.0-1 librdkafka1=1.1.0~1confluent5.3.0-1 librdkafka++1=1.1.0~1confluent5.3.0-1
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 librdkafka1 : Depends: libssl1.0.0 (>= 1.0.0) but it is not installable

Checklist

Please provide the following information:

confluent-kafka-go and librdkafka version (LibraryVersion()): attempting to install 1.1.0~1confluent5.3.0-1 of librdkafka
Apache Kafka broker version: n/a
Client configuration: ConfigMap{...}: n/a
Operating system: Debian Stretch
Provide client logs (with "debug": ".." as necessary): n/a
Provide broker log excerpts: n/a
Critical issue

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 2
Comments: 15 (5 by maintainers)

Most upvoted comments

We’re looking to release confluent-kafka-go with embedded librdkafka during Q1.

For now you should install librdkafka through one of the confluent apt/rpm repositories, homebrew, or a source install from github.

+12

edenhill on Feb 4, 2020

Hi @edenhill wanted to follow up on this. It doesn’t appear that static builds of the librdkafka library are supported yet in this official repo. Do you have a revised timeline?

I’m still experiencing confusion among teams I work with using this library for the first time expecting to have the same version of librdkafka as the version of confluent-kafka-go that they install for building their clients.

Many developers experience this in their builds, both in Debian and Alpine distros:

# gopkg.in/confluentinc/confluent-kafka-go.v1/kafka
/go/src/gopkg.in/confluentinc/confluent-kafka-go.v1/kafka/00version.go:44:2: error: #error "confluent-kafka-go requires librdkafka v1.3.0 or later. Install the latest version of librdkafka from the Confluent repositories, see http://docs.confluent.io/current/installation.html"
 #error "confluent-kafka-go requires librdkafka v1.3.0 or later. Install the latest version of librdkafka from the Confluent repositories, see http://docs.confluent.io/current/installation.html"
  ^~~~~

And then the documentation in http://docs.confluent.io/current/installation.html isn’t clear about how to resolve this dependency.

natemurthy on Jan 13, 2020

@natemurthy Happy to hear that it is working for you!

Answers:

confluent-kafka-go-dev is a proof of concept, it comes with no guarantees and should not be used in production.
we’re looking to merge the static builds into the official confluent-kafka-go repo during Q4. This makes the static builds a supported feature.

edenhill on Oct 7, 2019

when you rebuild librdkafka for recent debian/ubuntu (on withch the source build work fine with the openssl installed version that are more recent than 1.0…) ? because when we need c lib for kafka, we need c lib (not go lib)… if we need go lib we use go lib…

mcarbonneaux on Feb 1, 2021

Source builds are fine, it will use the OpenSSL version installed on the system. The problem is with the Debian packages that are built for an older Debian distro which has an openssl version which is not available on newer Debian distros, thus making the librdkafka packages uninstallable due to missing dependency.

edenhill on Aug 15, 2019

This is a known issue, the Confluent librdkafka debian packages are not compatible with Ubuntu >= 18., et.al, due to libssl1.0.0 not being available anymore.

There is no immediate fix in the pipeline for the debian package, but we’re looking at bundling librdkafka with the Go client to void the need for librdkafka installed separately. You can try it out by changing your confluent-kafka-go import to confluent-kafka-go-dev, see this example: https://github.com/confluentinc/confluent-kafka-go-example/blob/master/main.go#L23

That is all experimental at this point, but the more feedback we get the sooner it will land in the proper confluent-kafka-go repo.

edenhill on Aug 2, 2019

is this libssl conflict also the reason why 1.1.0 isn’t available for buster, bullseye, or sid?

https://packages.debian.org/sid/librdkafka-dev

Yes.

lizthegrey on Aug 9, 2019