confluent-kafka-dotnet: Confluent.Kafka cannot find librdkafka when running from dotnetcore 2.0 in docker

sadkafka.zip Description

Dotnet core 2 app cannot find librdkafka when running in docker image derived from the microsoft/dotnet base image

How to reproduce

Code attached.

Ensure you have dotnet core sdk 2 installed
Extract attached arcive
dotnet publish sadkafka.csproj (if you want; I already built it)
docker build --tag sadkafka .

If you were to navigate to /bin/Debug/netcoreapp2.0/publish and exec dotnet sadkafka.dll, everything works fine, however if you run the following

‘docker run sadkafka’

You should see the following stacktrace


zopadev@ubuntu:/media/sf_git/kafkaschmafka$ docker run sadkafka
Running...
Unhandled Exception: System.TypeInitializationException: The type initializer for 'Confluent.Kafka.Impl.LibRdKafka' threw an exception. ---> System.DllNotFoundException: Unable to load DLL 'librdkafka': The specified module or one of its dependencies could not be found.
 (Exception from HRESULT: 0x8007007E)
   at Confluent.Kafka.Impl.LibRdKafka.NativeMethods.rd_kafka_version()
   at Confluent.Kafka.Impl.LibRdKafka..cctor()
   --- End of inner exception stack trace ---
   at Confluent.Kafka.Impl.LibRdKafka.conf_new()
   at Confluent.Kafka.Impl.SafeConfigHandle.Create()
   at Confluent.Kafka.Producer..ctor(IEnumerable`1 config, Boolean manualPoll, Boolean disableDeliveryReports)
   at Confluent.Kafka.Producer`2..ctor(IEnumerable`1 config, ISerializer`1 keySerializer, ISerializer`1 valueSerializer, Boolean manualPoll, Boolean disableDeliveryReports)
   at sadkafka.Program.Main(String[] args) in D:\git\kafkaschmafka\Program.cs:line 22

Checklist

Please provide the following information:

Confluent.Kafka nuget version: 0.11.0
Apache Kafka version: na
Client configuration: na
Operating system: ubuntu, windows (works) vs docker: microsoft/dotnet (based on debian:stretch)
Provide logs (with “debug” : “…” as necessary in configuration) NA
Provide broker log excerpts NA
Critical issue Its blocking us (and we have a support contract with you)

About this issue

Original URL
State: closed
Created 7 years ago
Comments: 33 (16 by maintainers)

Most upvoted comments

Currently working on it. I’m re-packaging librdkafka.redist and making some changes to the dotnet library to make native library loading more flexible. I thought it would be done by now but it’s turned out to be quite difficult.

mhowlett on Oct 22, 2017

@spadger Use microsoft/dotnet:2-runtime-jessie or 2-sdk-jessie in your docker file, this will work (for reference, this comes from dependencies installed here: https://github.com/dotnet/dotnet-docker/blob/master/2.0/sdk/jessie/amd64/Dockerfile https://github.com/dotnet/dotnet-docker/blob/master/2.0/sdk/stretch/amd64/Dockerfile )

treziac on Aug 17, 2017

Hey guys, I was getting this error with microsoft/dotnet:2-runtime. Switching to the microsoft/dotnet:2-runtime-jessie image suggested by @treziac, seemed to solve the problem.

Now, a couple weeks later, I just updated another project to dotnet core 2.0, and even with the 2-runtime-jessie image, I’m seeing this issue.

I got a shell into the container and ran a search for the libraries listed in one of @mhowlett’s posts above and these are the paths I found:

    /lib/x86_64-linux-gnu/libpthread.so.0
    /lib/x86_64-linux-gnu/libz.so.1
    /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
    /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0
    /usr/lib/x86_64-linux-gnu/libsasl2.so.2
    /lib/x86_64-linux-gnu/librt.so.1
    /lib/x86_64-linux-gnu/libdl.so.2
    /lib64/ld-linux-x86-64.so.2
    /lib/x86_64-linux-gnu/libresolv.so.2

GarrettClyde on Oct 24, 2017

Thanks @AndreSteenbergen An explicit reference to librdkafka.redist wasn’t my specific issue, but your comment got me looking in the right place. It looks like this package behaves differently depending on the runtime identifier used in the dotnet publish command.

I was having issues with my Kubernetes cluster and the dotnet debian.8-x64 RID so I changed it to the new linux-x64 RID.

When publishing projects using either debian-x64 or debian.8-x64, I’m seeing

    out/
    ├── ...
    ├── librdkafka++.so
    ├── librdkafka.so
    └── x64/
        ├── librdkafka.dll
        ├── librdkafka.lib
        ├── librdkafkacpp.dll
        ├── librdkafkacpp.lib
        └── ...

When publishing using --runtime linux-x64, I’m not seeing the .so files

    out/
    ├── ...
    └── x64/
        ├── librdkafka.dll
        ├── librdkafka.lib
        ├── librdkafkacpp.dll
        ├── librdkafkacpp.lib
        └── ...

For anyone else running into this issue, changing my RID to debian-x64 allowed my container to run happily in kops-managed Kubernetes cluster with the confluent-kafka-dotnet library.

GarrettClyde on Oct 26, 2017

Well, I did some more testing (I could not replicate what I sais first, I may have mixed something) I still don’t understand - even the text does you paste @mhowlett does not explain it - the runtimes are still here, and librdkafka.so is well copied in output folder. Either a dependencies is missing (but which one?) either dllImport has changed. Will try to look a bit more on this tomorrow as we need this at work

treziac on Aug 16, 2017