node-rdkafka: Mac build is broken

Thanks to https://github.com/Blizzard/node-rdkafka/issues/405#issuecomment-398139737, I solved this on my mac.

I set BUILD_LIBRDKAFKA to 0, to prevent node-gyp build the dependencies. Then I used homebrew to install the kafka bindings brew install librdkafka

brew install librdkafka
BUILD_LIBRDKAFKA=0
yarn install

My error was:

  "_X509_verify_cert_error_string", referenced from:
      _rd_kafka_transport_io_serve in rdkafka_transport.o
  "_d2i_PKCS12_fp", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
  "_sk_pop_free", referenced from:
      _rd_kafka_transport_ssl_ctx_init in rdkafka_transport.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [librdkafka.1.dylib] Error 1
make[1]: *** [libs] Error 2
make: *** [_Users_qrpike_project_node_modules_node_rdkafka_deps_librdkafka_gyp_librdkafka_target_build_dependencies.intermediate] Error 2
rm _Users_qrpike_project_node_modules_node_rdkafka_deps_librdkafka_gyp_librdkafka_target_build_dependencies.intermediate
000: qrpike$ ld```

I face the same issue while installing node-rdkafka on Ubuntu from master using yarn.

yarn add https://github.com/Blizzard/node-rdkafka

Could you trigger new release, so that your package can be installed from npm?

i have solve that erros: ⇒ echo ‘export PATH=“/usr/local/opt/openssl/bin:$PATH”’ >> ~/.zshrc ⇒ echo ‘export LDFLAGS=“-L/usr/local/opt/openssl/lib”’ >> ~/.zshrc ⇒ echo ‘export CPPFLAGS=“-I/usr/local/opt/openssl/include”’ >> ~/.zshrc ⇒ reload terminal

but i got new error:

Release/node-librdkafka.node clang: error: no such file or directory: '/Users/wangyong/nodejs/jd-banner-kafka/node_modules/node-rdkafka/build/deps/librdkafka.dylib' clang: error: no such file or directory: '/Users/wangyong/nodejs/jd-banner-kafka/node_modules/node-rdkafka/build/deps/librdkafka++.dylib'

Yes, now I’m getting different error. Looks like ./configure is missing

$ yarn add https://github.com/Blizzard/node-rdkafka.git
yarn add v1.5.1
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
[1/1] ⠄ node-rdkafka: ./configure: line 13: ./configure: No such file or directory
[-/1] ⠄ waiting...
[-/1] ⠄ waiting...
[-/1] ⠄ waiting...
error An unexpected error occurred: "<path_to_app>/node_modules/node-rdkafka: Command failed.
Exit code: 1
Command: sh
Arguments: -c node-gyp rebuild
Directory: <path_to_app>/node_modules/node-rdkafka
Output:
gyp info it worked if it ends with ok
gyp info using node-gyp@3.6.2
gyp info using node@9.11.1 | darwin | x64
gyp info spawn /usr/bin/python
gyp info spawn args [ '/usr/local/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '<path_to_app>/node_modules/node-rdkafka/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/usr/local/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '~/.node-gyp/9.11.1/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=~/.node-gyp/9.11.1',
gyp info spawn args   '-Dnode_gyp_dir=/usr/local/lib/node_modules/npm/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=~/.node-gyp/9.11.1/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=<path_to_app>/node_modules/node-rdkafka',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.' ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
  ACTION path_to_app_node_modules_node_rdkafka_deps_librdkafka_gyp_librdkafka_target_configure <path_to_app>/node_modules/node-rdkafka/deps/librdkafka/config.h
./configure: line 13: ./configure: No such file or directory
  TOUCH path_to_app_node_modules_node_rdkafka_deps_librdkafka_gyp_librdkafka_target_build_dependencies.intermediate
  ACTION path_to_app_node_modules_node_rdkafka_deps_librdkafka_gyp_librdkafka_target_build_dependencies path_to_app_node_modules_node_rdkafka_deps_librdkafka_gyp_librdkafka_target_build_dependencies.intermediate
make[1]: *** No rule to make target `libs'.  Stop.
make: *** [path_to_app_node_modules_node_rdkafka_deps_librdkafka_gyp_librdkafka_target_build_dependencies.intermediate] Error 2
rm path_to_app_node_modules_node_rdkafka_deps_librdkafka_gyp_librdkafka_target_build_dependencies.intermediate
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:258:23)
gyp ERR! stack     at ChildProcess.emit (events.js:180:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:209:12)
gyp ERR! System Darwin 17.5.0
gyp ERR! command \"/usr/local/bin/node\" \"/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js\" \"rebuild\"
gyp ERR! cwd <path_to_app>/node_modules/node-rdkafka
gyp ERR! node -v v9.11.1

This is looking more and more to me like an NPM bug. Or maybe I was just using pre-install in a way it wasn’t intended.

It makes this staging directory and then has the node-gyp variables resolve to the staging directory during the configure step. At that point, it moves the files into the correct directory, so when it tries to link it gets messed up because the prefix was made at preinstall time.

I made an adjustment to put this in gyp again. Can you try to build this from github now? You should (hopefully) at least be getting a different error