bazel-gazelle: Cannot use Gazelle to build confluent-kafka-go

What version of gazelle are you using?

v0.20.0

What version of rules_go are you using?

v0.22.4

What version of Bazel are you using?

3.1.0

Does this issue reproduce with the latest releases of all the above?

Yes

What operating system and processor architecture are you using?

OSX x86_64

What did you do?

git clone https://github.com/confluentinc/confluent-kafka-go
Add the root-level WORKSPACE and BUILD files as shown on the bazel-gazelle readme to the directory.
Run bazel run //:gazelle
Run `bazel build //kafka:go_default_library

What did you expect to see?

The confluent kafka client to be built successfully.

What did you see instead?

▶ bazel build --verbose_failures --sandbox_debug //kafka:go_default_library
INFO: Analyzed target //kafka:go_default_library (2 packages loaded, 34 targets configured).
INFO: Found 1 target...
ERROR: /Users/epaulson/repos/confluent-kafka-go/kafka/BUILD.bazel:3:1: GoCompilePkg kafka/darwin_amd64_stripped/go_default_library%/github.com/confluentinc/confluent-kafka-go/kafka.a failed (Exit 1): sandbox-exec failed: error executing command
  (cd /private/var/tmp/_bazel_epaulson/6b12ee4ca700c7be6f2e76362bbd93f7/sandbox/darwin-sandbox/68/execroot/__main__ && \
  exec env - \
    APPLE_SDK_PLATFORM=MacOSX \
    APPLE_SDK_VERSION_OVERRIDE=10.14 \
    CC=external/local_config_cc/wrapped_clang \
    CGO_ENABLED=1 \
    DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer \
    GOARCH=amd64 \
    GOOS=darwin \
    GOPATH='' \
    GOROOT=external/go_sdk \
    GOROOT_FINAL=GOROOT \
    PATH=external/local_config_cc:/bin:/usr/bin \
    SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk \
    TMPDIR=/var/folders/9f/vj80rvq96tgfh3st53b5f0rs5vf503/T/ \
    XCODE_VERSION_OVERRIDE=10.3.0.10G8 \
  /usr/bin/sandbox-exec -f /private/var/tmp/_bazel_epaulson/6b12ee4ca700c7be6f2e76362bbd93f7/sandbox/darwin-sandbox/68/sandbox.sb /var/tmp/_bazel_epaulson/install/1d6d7a22a62da56414387a04bacbf619/process-wrapper '--timeout=0' '--kill_delay=15' bazel-out/host/bin/external/go_sdk/builder compilepkg -sdk external/go_sdk -installsuffix darwin_amd64 -src kafka/00version.go -src kafka/adminapi.go -src kafka/adminoptions.go -src kafka/build_darwin.go -src kafka/build_glibc_linux.go -src kafka/config.go -src kafka/consumer.go -src kafka/context.go -src kafka/error.go -src kafka/error_gen.go -src kafka/event.go -src kafka/generated_errors.go -src kafka/handle.go -src kafka/header.go -src kafka/kafka.go -src kafka/log.go -src kafka/message.go -src kafka/metadata.go -src kafka/misc.go -src kafka/offset.go -src kafka/producer.go -src kafka/testhelpers.go -src kafka/time.go -src kafka/glue_rdkafka.h -arc 'github.com/confluentinc/confluent-kafka-go/kafka/librdkafka=github.com/confluentinc/confluent-kafka-go/kafka/librdkafka=bazel-out/darwin-fastbuild/bin/kafka/librdkafka/darwin_amd64_stripped/go_default_library%/github.com/confluentinc/confluent-kafka-go/kafka/librdkafka.a=' -importpath github.com/confluentinc/confluent-kafka-go/kafka -p github.com/confluentinc/confluent-kafka-go/kafka -package_list bazel-out/host/bin/external/go_sdk/packages.txt -o bazel-out/darwin-fastbuild/bin/kafka/darwin_amd64_stripped/go_default_library%/github.com/confluentinc/confluent-kafka-go/kafka.a -gcflags '' -asmflags '' -cppflags '-I kafka -iquote kafka -iquote .' -cflags '-D_FORTIFY_SOURCE=1 -fstack-protector -Wthread-safety -Wself-assign -fno-omit-frame-pointer -O0 -DDEBUG -isysroot __BAZEL_XCODE_SDKROOT__ -F__BAZEL_XCODE_SDKROOT__/System/Library/Frameworks -F__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/Library/Frameworks -mmacosx-version-min=10.14 -no-canonical-prefixes -Wno-builtin-macro-redefined -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted" -Ikafka/kafka -fPIC' -cxxflags '-D_FORTIFY_SOURCE=1 -fstack-protector -Wthread-safety -Wself-assign -fno-omit-frame-pointer -O0 -DDEBUG -std=c++11 -isysroot __BAZEL_XCODE_SDKROOT__ -F__BAZEL_XCODE_SDKROOT__/System/Library/Frameworks -F__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/Library/Frameworks -mmacosx-version-min=10.14 -no-canonical-prefixes -Wno-builtin-macro-redefined -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted" -fPIC' -objcflags '-D_FORTIFY_SOURCE=1 -fstack-protector -Wthread-safety -Wself-assign -fno-omit-frame-pointer -O0 -DDEBUG -isysroot __BAZEL_XCODE_SDKROOT__ -F__BAZEL_XCODE_SDKROOT__/System/Library/Frameworks -F__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/Library/Frameworks -mmacosx-version-min=10.14 -D_FORTIFY_SOURCE=1 -fstack-protector -Wthread-safety -Wself-assign -fno-omit-frame-pointer -O0 -DDEBUG -isysroot __BAZEL_XCODE_SDKROOT__ -F__BAZEL_XCODE_SDKROOT__/System/Library/Frameworks -F__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/Library/Frameworks -mmacosx-version-min=10.14 -no-canonical-prefixes -Wno-builtin-macro-redefined -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted" -Ikafka/kafka -fPIC' -objcxxflags '-D_FORTIFY_SOURCE=1 -fstack-protector -Wthread-safety -Wself-assign -fno-omit-frame-pointer -O0 -DDEBUG -isysroot __BAZEL_XCODE_SDKROOT__ -F__BAZEL_XCODE_SDKROOT__/System/Library/Frameworks -F__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/Library/Frameworks -mmacosx-version-min=10.14 -D_FORTIFY_SOURCE=1 -fstack-protector -Wthread-safety -Wself-assign -fno-omit-frame-pointer -O0 -DDEBUG -std=c++11 -isysroot __BAZEL_XCODE_SDKROOT__ -F__BAZEL_XCODE_SDKROOT__/System/Library/Frameworks -F__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/Library/Frameworks -mmacosx-version-min=10.14 -no-canonical-prefixes -Wno-builtin-macro-redefined -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted" -fPIC' -ldflags '-lc++ -fobjc-link-runtime -headerpad_max_install_names -no-canonical-prefixes -mmacosx-version-min=10.14 kafka/librdkafka/librdkafka_darwin.a -lm -lsasl2 -lz -ldl -lpthread') sandbox-exec failed: error executing command
  (cd /private/var/tmp/_bazel_epaulson/6b12ee4ca700c7be6f2e76362bbd93f7/sandbox/darwin-sandbox/68/execroot/__main__ && \
  exec env - \
    APPLE_SDK_PLATFORM=MacOSX \
    APPLE_SDK_VERSION_OVERRIDE=10.14 \
    CC=external/local_config_cc/wrapped_clang \
    CGO_ENABLED=1 \
    DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer \
    GOARCH=amd64 \
    GOOS=darwin \
    GOPATH='' \
    GOROOT=external/go_sdk \
    GOROOT_FINAL=GOROOT \
    PATH=external/local_config_cc:/bin:/usr/bin \
    SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk \
    TMPDIR=/var/folders/9f/vj80rvq96tgfh3st53b5f0rs5vf503/T/ \
    XCODE_VERSION_OVERRIDE=10.3.0.10G8 \
  /usr/bin/sandbox-exec -f /private/var/tmp/_bazel_epaulson/6b12ee4ca700c7be6f2e76362bbd93f7/sandbox/darwin-sandbox/68/sandbox.sb /var/tmp/_bazel_epaulson/install/1d6d7a22a62da56414387a04bacbf619/process-wrapper '--timeout=0' '--kill_delay=15' bazel-out/host/bin/external/go_sdk/builder compilepkg -sdk external/go_sdk -installsuffix darwin_amd64 -src kafka/00version.go -src kafka/adminapi.go -src kafka/adminoptions.go -src kafka/build_darwin.go -src kafka/build_glibc_linux.go -src kafka/config.go -src kafka/consumer.go -src kafka/context.go -src kafka/error.go -src kafka/error_gen.go -src kafka/event.go -src kafka/generated_errors.go -src kafka/handle.go -src kafka/header.go -src kafka/kafka.go -src kafka/log.go -src kafka/message.go -src kafka/metadata.go -src kafka/misc.go -src kafka/offset.go -src kafka/producer.go -src kafka/testhelpers.go -src kafka/time.go -src kafka/glue_rdkafka.h -arc 'github.com/confluentinc/confluent-kafka-go/kafka/librdkafka=github.com/confluentinc/confluent-kafka-go/kafka/librdkafka=bazel-out/darwin-fastbuild/bin/kafka/librdkafka/darwin_amd64_stripped/go_default_library%/github.com/confluentinc/confluent-kafka-go/kafka/librdkafka.a=' -importpath github.com/confluentinc/confluent-kafka-go/kafka -p github.com/confluentinc/confluent-kafka-go/kafka -package_list bazel-out/host/bin/external/go_sdk/packages.txt -o bazel-out/darwin-fastbuild/bin/kafka/darwin_amd64_stripped/go_default_library%/github.com/confluentinc/confluent-kafka-go/kafka.a -gcflags '' -asmflags '' -cppflags '-I kafka -iquote kafka -iquote .' -cflags '-D_FORTIFY_SOURCE=1 -fstack-protector -Wthread-safety -Wself-assign -fno-omit-frame-pointer -O0 -DDEBUG -isysroot __BAZEL_XCODE_SDKROOT__ -F__BAZEL_XCODE_SDKROOT__/System/Library/Frameworks -F__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/Library/Frameworks -mmacosx-version-min=10.14 -no-canonical-prefixes -Wno-builtin-macro-redefined -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted" -Ikafka/kafka -fPIC' -cxxflags '-D_FORTIFY_SOURCE=1 -fstack-protector -Wthread-safety -Wself-assign -fno-omit-frame-pointer -O0 -DDEBUG -std=c++11 -isysroot __BAZEL_XCODE_SDKROOT__ -F__BAZEL_XCODE_SDKROOT__/System/Library/Frameworks -F__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/Library/Frameworks -mmacosx-version-min=10.14 -no-canonical-prefixes -Wno-builtin-macro-redefined -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted" -fPIC' -objcflags '-D_FORTIFY_SOURCE=1 -fstack-protector -Wthread-safety -Wself-assign -fno-omit-frame-pointer -O0 -DDEBUG -isysroot __BAZEL_XCODE_SDKROOT__ -F__BAZEL_XCODE_SDKROOT__/System/Library/Frameworks -F__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/Library/Frameworks -mmacosx-version-min=10.14 -D_FORTIFY_SOURCE=1 -fstack-protector -Wthread-safety -Wself-assign -fno-omit-frame-pointer -O0 -DDEBUG -isysroot __BAZEL_XCODE_SDKROOT__ -F__BAZEL_XCODE_SDKROOT__/System/Library/Frameworks -F__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/Library/Frameworks -mmacosx-version-min=10.14 -no-canonical-prefixes -Wno-builtin-macro-redefined -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted" -Ikafka/kafka -fPIC' -objcxxflags '-D_FORTIFY_SOURCE=1 -fstack-protector -Wthread-safety -Wself-assign -fno-omit-frame-pointer -O0 -DDEBUG -isysroot __BAZEL_XCODE_SDKROOT__ -F__BAZEL_XCODE_SDKROOT__/System/Library/Frameworks -F__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/Library/Frameworks -mmacosx-version-min=10.14 -D_FORTIFY_SOURCE=1 -fstack-protector -Wthread-safety -Wself-assign -fno-omit-frame-pointer -O0 -DDEBUG -std=c++11 -isysroot __BAZEL_XCODE_SDKROOT__ -F__BAZEL_XCODE_SDKROOT__/System/Library/Frameworks -F__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/Library/Frameworks -mmacosx-version-min=10.14 -no-canonical-prefixes -Wno-builtin-macro-redefined -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted" -fPIC' -ldflags '-lc++ -fobjc-link-runtime -headerpad_max_install_names -no-canonical-prefixes -mmacosx-version-min=10.14 kafka/librdkafka/librdkafka_darwin.a -lm -lsasl2 -lz -ldl -lpthread')
/private/var/tmp/_bazel_epaulson/6b12ee4ca700c7be6f2e76362bbd93f7/sandbox/darwin-sandbox/68/execroot/__main__/kafka/00version.go:24:10: fatal error: 'librdkafka/rdkafka.h' file not found
#include <librdkafka/rdkafka.h>
         ^~~~~~~~~~~~~~~~~~~~~~
1 error generated.
compilepkg: error running subcommand: exit status 2
Target //kafka:go_default_library failed to build
INFO: Elapsed time: 0.475s, Critical Path: 0.26s
INFO: 0 processes.
FAILED: Build did NOT complete successfully

What I think is happening

This repository uses cgo, and has these directives in build_darwin.go:

// #cgo CFLAGS: -I${SRCDIR}
// #cgo LDFLAGS: ${SRCDIR}/librdkafka/librdkafka_darwin.a  -lm -lsasl2 -lz -ldl -lpthread

According to CGO documentation the SRCDIR directive is expanded to an absolute path, yet Gazelle is expanding it to a relative path as you can see in the -cflags argument in the output above: -Ikafka/kafka. This is odd because not only is that a relative directory, but kafka/kafka does not exist. This similarly is happening for the library to link.

If I manually edit the BUILD.bazel file generated by Gazelle to use absolute paths for these includes and libraries to link, it builds correctly.

Is there agreement that ${SRCDIR} should expand to an absolute path? If so, I would be happy to take a crack at it and submit a PR.

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 19 (7 by maintainers)

Most upvoted comments

Sadly v0.25 of rules_go did not fix the issue. Here’s an updated patch that’s working with bazel 4.0.0, confluent-kafka-go 1.7.0, bazel-gazelle v0.23.0 and rules-go v0.27.0. The folder structure has been updated, and there’s a new rule to compile for windows as well.

This is for those using github.com/confluentinc/confluent-kafka-go for their imports.

Those using gopkg.in/confluentinc/confluent-kafka-go.v1 for their imports, replace all the occurrences of com_github_confluentinc_confluent_kafka_go with in_gopkg_confluentinc_confluent_kafka_go_v1, or whatever is in the name parameter of your confluent-kafka-go go_repository rule.

--- kafka/librdkafka_vendor/BUILD.bazel
+++ kafka/librdkafka_vendor/BUILD.bazel
@@ -1,5 +1,27 @@
 load("@io_bazel_rules_go//go:def.bzl", "go_library")

+cc_library(
+    name = "librdkafka_static",
+    srcs = select({
+        "@io_bazel_rules_go//go/platform:android": [
+            "librdkafka_glibc_linux.a",
+        ],
+        "@io_bazel_rules_go//go/platform:darwin": [
+            "librdkafka_darwin.a",
+        ],
+        "@io_bazel_rules_go//go/platform:ios": [
+            "librdkafka_darwin.a",
+        ],
+        "@io_bazel_rules_go//go/platform:linux": [
+            "librdkafka_glibc_linux.a",
+            "librdkafka_musl_linux.a",
+        ],
+        "//conditions:default": [],
+    }),
+    hdrs = ["rdkafka.h"],
+    visibility = ["//visibility:public"],
+)
+
 go_library(
     name = "librdkafka_vendor",
     srcs = [

--- kafka/BUILD.bazel
+++ kafka/BUILD.bazel
@@ -31,21 +31,22 @@
         "time.go",
     ],
     cgo = True,
+    cdeps = ["//kafka/librdkafka_vendor:librdkafka_static"],
     clinkopts = select({
         "@io_bazel_rules_go//go/platform:android": [
-            "kafka/librdkafka_vendor/librdkafka_glibc_linux.a -lm -ldl -lpthread -lrt",
+            "external/com_github_confluentinc_confluent_kafka_go/kafka/librdkafka_vendor/librdkafka_glibc_linux.a -lm -ldl -lpthread -lrt",
         ],
         "@io_bazel_rules_go//go/platform:darwin": [
-            "kafka/librdkafka_vendor/librdkafka_darwin.a -lz -lm -lsasl2 -ldl -lpthread",
+            "external/com_github_confluentinc_confluent_kafka_go/kafka/librdkafka_vendor/librdkafka_darwin.a -lz -lm -lsasl2 -ldl -lpthread",
         ],
         "@io_bazel_rules_go//go/platform:ios": [
-            "kafka/librdkafka_vendor/librdkafka_darwin.a -lz -lm -lsasl2 -ldl -lpthread",
+            "external/com_github_confluentinc_confluent_kafka_go/kafka/librdkafka_vendor/librdkafka_darwin.a -lz -lm -lsasl2 -ldl -lpthread",
         ],
         "@io_bazel_rules_go//go/platform:linux": [
-            "kafka/librdkafka_vendor/librdkafka_glibc_linux.a -lm -ldl -lpthread -lrt",
+            "external/com_github_confluentinc_confluent_kafka_go/kafka/librdkafka_vendor/librdkafka_glibc_linux.a -lm -ldl -lpthread -lrt",
         ],
         "@io_bazel_rules_go//go/platform:windows": [
-            "kafka/librdkafka_vendor/librdkafka_windows.a -lws2_32 -lsecur32 -lcrypt32",
+            "external/com_github_confluentinc_confluent_kafka_go/kafka/librdkafka_vendor/librdkafka_windows.a -lws2_32 -lsecur32 -lcrypt32",
         ],
         "//conditions:default": [],
     }),

+12

radneran on Jul 13, 2021

For v1.9.1, librdkafka_darwin.a has been replaced by librdkafka_darwin_amd64.a and librdkafka_darwin_arm64.a, this patch worked for me:

--- kafka/librdkafka_vendor/BUILD.bazel
+++ kafka/librdkafka_vendor/BUILD.bazel
@@ -1,5 +1,33 @@
 load("@io_bazel_rules_go//go:def.bzl", "go_library")
 
+cc_library(
+    name = "librdkafka_static",
+    srcs = select({
+        "@io_bazel_rules_go//go/platform:android": [
+            "librdkafka_glibc_linux.a",
+        ],
+        "@io_bazel_rules_go//go/platform:darwin_amd64": [
+            "librdkafka_darwin_amd64.a",
+        ],
+        "@io_bazel_rules_go//go/platform:darwin_arm64": [
+            "librdkafka_darwin_arm64.a",
+        ],
+        "@io_bazel_rules_go//go/platform:ios_amd64": [
+            "librdkafka_darwin_amd64.a",
+        ],
+        "@io_bazel_rules_go//go/platform:ios_arm64": [
+            "librdkafka_darwin_arm64.a"
+        ],
+        "@io_bazel_rules_go//go/platform:linux": [
+            "librdkafka_glibc_linux.a",
+            "librdkafka_musl_linux.a",
+        ],
+        "//conditions:default": [],
+    }),
+    hdrs = ["rdkafka.h", "rdkafka_mock.h"],
+    visibility = ["//visibility:public"],
+)
+
 go_library(
     name = "librdkafka_vendor",
     srcs = [

--- kafka/BUILD.bazel
+++ kafka/BUILD.bazel
@@ -33,29 +33,30 @@
         "time.go",
     ],
     cgo = True,
+    cdeps = ["//kafka/librdkafka_vendor:librdkafka_static"],
     clinkopts = select({
         "@io_bazel_rules_go//go/platform:android": [
-            "kafka/librdkafka_vendor/librdkafka_glibc_linux.a -lm -ldl -lpthread -lrt",
+            "external/com_github_confluentinc_confluent_kafka_go/kafka/librdkafka_vendor/librdkafka_glibc_linux.a -lm -ldl -lpthread -lrt",
         ],
         "@io_bazel_rules_go//go/platform:linux": [
-            "kafka/librdkafka_vendor/librdkafka_glibc_linux.a -lm -ldl -lpthread -lrt",
+            "external/com_github_confluentinc_confluent_kafka_go/kafka/librdkafka_vendor/librdkafka_glibc_linux.a -lm -ldl -lpthread -lrt",
         ],
         "@io_bazel_rules_go//go/platform:windows": [
-            "kafka/librdkafka_vendor/librdkafka_windows.a -lws2_32 -lsecur32 -lcrypt32",
+            "external/com_github_confluentinc_confluent_kafka_go/kafka/librdkafka_vendor/librdkafka_windows.a -lws2_32 -lsecur32 -lcrypt32",
         ],
         "//conditions:default": [],
     }) + select({
         "@io_bazel_rules_go//go/platform:darwin_amd64": [
-            "kafka/librdkafka_vendor/librdkafka_darwin_amd64.a -lm -lsasl2 -ldl -lpthread -framework CoreFoundation -framework SystemConfiguration",
+            "external/com_github_confluentinc_confluent_kafka_go/kafka/librdkafka_vendor/librdkafka_darwin_amd64.a -lm -lsasl2 -ldl -lpthread -framework CoreFoundation -framework SystemConfiguration",
         ],
         "@io_bazel_rules_go//go/platform:darwin_arm64": [
-            "kafka/librdkafka_vendor/librdkafka_darwin_arm64.a -lm -lsasl2 -ldl -lpthread -framework CoreFoundation -framework SystemConfiguration",
+            "external/com_github_confluentinc_confluent_kafka_go/kafka/librdkafka_vendor/librdkafka_darwin_arm64.a -lm -lsasl2 -ldl -lpthread -framework CoreFoundation -framework SystemConfiguration",
         ],
         "@io_bazel_rules_go//go/platform:ios_amd64": [
-            "kafka/librdkafka_vendor/librdkafka_darwin_amd64.a -lm -lsasl2 -ldl -lpthread -framework CoreFoundation -framework SystemConfiguration",
+            "external/com_github_confluentinc_confluent_kafka_go/kafka/librdkafka_vendor/librdkafka_darwin_amd64.a -lm -lsasl2 -ldl -lpthread -framework CoreFoundation -framework SystemConfiguration",
         ],
         "@io_bazel_rules_go//go/platform:ios_arm64": [
-            "kafka/librdkafka_vendor/librdkafka_darwin_arm64.a -lm -lsasl2 -ldl -lpthread -framework CoreFoundation -framework SystemConfiguration",
+            "external/com_github_confluentinc_confluent_kafka_go/kafka/librdkafka_vendor/librdkafka_darwin_arm64.a -lm -lsasl2 -ldl -lpthread -framework CoreFoundation -framework SystemConfiguration",
         ],
         "//conditions:default": [],
     }),

jgao54 on Aug 1, 2022

Geez, as a complete bazel noob, this is really hard to follow. Can anyone break down the steps a bit more clearly as if you were explaining this to a 5 year old? How do you create bazel patches? Where does the patch go to be used? etc

cluix on Dec 31, 2022

For v1.9.0, I had to add rdkafka_mock.h to the hdrs list on the patch provided by @radneran

darkrift on Jul 18, 2022

Ok, we figured out an way without having to compile librdkafka. If you are using Bazel 4.0.0 or later you will have to use rules_go 0.25(unreleased at the time of writing) or later to fix this issue: https://github.com/bazelbuild/rules_go/issues/2720

This is for version 1.5.2 of confluent-kafka-go

Patch if you import gopkg.in/confluentinc/confluent-kafka-go.v1 in your app

--- kafka/librdkafka/BUILD.bazel
+++ kafka/librdkafka/BUILD.bazel
@@ -1,5 +1,27 @@
 load("@io_bazel_rules_go//go:def.bzl", "go_library")
 
+cc_library(
+    name = "librdkafka_static",
+    srcs = select({
+        "@io_bazel_rules_go//go/platform:android": [
+            "librdkafka_glibc_linux.a",
+        ],
+        "@io_bazel_rules_go//go/platform:darwin": [
+            "librdkafka_darwin.a",
+        ],
+        "@io_bazel_rules_go//go/platform:ios": [
+            "librdkafka_darwin.a",
+        ],
+        "@io_bazel_rules_go//go/platform:linux": [
+            "librdkafka_glibc_linux.a",
+            "librdkafka_musl_linux.a",
+        ],
+        "//conditions:default": [],
+    }),
+    hdrs = ["rdkafka.h"],
+    visibility = ["//visibility:public"],
+)
+
 go_library(
     name = "librdkafka",
     srcs = [

--- kafka/BUILD.bazel
+++ kafka/BUILD.bazel
@@ -28,19 +28,20 @@
         "testhelpers.go",
         "time.go",
     ],
+    cdeps = ["//kafka/librdkafka:librdkafka_static"],
     cgo = True,
     clinkopts = select({
         "@io_bazel_rules_go//go/platform:android": [
-            "kafka/librdkafka/librdkafka_glibc_linux.a -lm -ldl -lpthread -lrt",
+            "external/in_gopkg_confluentinc_confluent_kafka_go_v1/kafka/librdkafka/librdkafka_glibc_linux.a -lm -ldl -lpthread -lrt",
         ],
         "@io_bazel_rules_go//go/platform:darwin": [
-            "kafka/librdkafka/librdkafka_darwin.a -lz -lm -lsasl2 -ldl -lpthread",
+            "external/in_gopkg_confluentinc_confluent_kafka_go_v1/kafka/librdkafka/librdkafka_darwin.a -lz -lm -lsasl2 -ldl -lpthread",
         ],
         "@io_bazel_rules_go//go/platform:ios": [
-            "kafka/librdkafka/librdkafka_darwin.a -lz -lm -lsasl2 -ldl -lpthread",
+            "external/in_gopkg_confluentinc_confluent_kafka_go_v1/kafka/librdkafka/librdkafka_darwin.a -lz -lm -lsasl2 -ldl -lpthread",
         ],
         "@io_bazel_rules_go//go/platform:linux": [
-            "kafka/librdkafka/librdkafka_glibc_linux.a -lm -ldl -lpthread -lrt",
+            "external/in_gopkg_confluentinc_confluent_kafka_go_v1/kafka/librdkafka/librdkafka_glibc_linux.a -lm -ldl -lpthread -lrt",
         ],
         "//conditions:default": [],
     }),

Patch if you import github.com/confluentinc/confluent-kafka-go in your code:

--- kafka/librdkafka/BUILD.bazel
+++ kafka/librdkafka/BUILD.bazel
@@ -1,5 +1,27 @@
 load("@io_bazel_rules_go//go:def.bzl", "go_library")
 
+cc_library(
+    name = "librdkafka_static",
+    srcs = select({
+        "@io_bazel_rules_go//go/platform:android": [
+            "librdkafka_glibc_linux.a",
+        ],
+        "@io_bazel_rules_go//go/platform:darwin": [
+            "librdkafka_darwin.a",
+        ],
+        "@io_bazel_rules_go//go/platform:ios": [
+            "librdkafka_darwin.a",
+        ],
+        "@io_bazel_rules_go//go/platform:linux": [
+            "librdkafka_glibc_linux.a",
+            "librdkafka_musl_linux.a",
+        ],
+        "//conditions:default": [],
+    }),
+    hdrs = ["rdkafka.h"],
+    visibility = ["//visibility:public"],
+)
+
 go_library(
     name = "librdkafka",
     srcs = [

--- kafka/BUILD.bazel
+++ kafka/BUILD.bazel
@@ -28,19 +28,20 @@
         "testhelpers.go",
         "time.go",
     ],
+    cdeps = ["//kafka/librdkafka:librdkafka_static"],
     cgo = True,
     clinkopts = select({
         "@io_bazel_rules_go//go/platform:android": [
-            "kafka/librdkafka/librdkafka_glibc_linux.a -lm -ldl -lpthread -lrt",
+            "external/com_github_confluentinc_confluent_kafka_go/kafka/librdkafka/librdkafka_glibc_linux.a -lm -ldl -lpthread -lrt",
         ],
         "@io_bazel_rules_go//go/platform:darwin": [
-            "kafka/librdkafka/librdkafka_darwin.a -lz -lm -lsasl2 -ldl -lpthread",
+            "external/com_github_confluentinc_confluent_kafka_go/kafka/librdkafka/librdkafka_darwin.a -lz -lm -lsasl2 -ldl -lpthread",
         ],
         "@io_bazel_rules_go//go/platform:ios": [
-            "kafka/librdkafka/librdkafka_darwin.a -lz -lm -lsasl2 -ldl -lpthread",
+            "external/com_github_confluentinc_confluent_kafka_go/kafka/librdkafka/librdkafka_darwin.a -lz -lm -lsasl2 -ldl -lpthread",
         ],
         "@io_bazel_rules_go//go/platform:linux": [
-            "kafka/librdkafka/librdkafka_glibc_linux.a -lm -ldl -lpthread -lrt",
+            "external/com_github_confluentinc_confluent_kafka_go/kafka/librdkafka/librdkafka_glibc_linux.a -lm -ldl -lpthread -lrt",
         ],
         "//conditions:default": [],
     }),

purkhusid on Feb 22, 2021