kafka-docker: Not works on Docker Mac Native - Zookeeper timeouts

I’m using Docker Native with 6GB RAM, 2CPU Core.

$ docker -v
Docker version 1.12.0, build 8eab29e

kafka_2.11-0.10.0.0, zookeeper-3.4.6

I try make steps in help for testing.

$ docker-compose up -d
$ docker-compose scale kafka=2
$ ./start-kafka-shell.sh 172.18.0.2 172.18.0.3:2181
bash-4.3# $KAFKA_HOME/bin/kafka-topics.sh --create --topic topic \
> --partitions 4 --zookeeper $ZK --replication-factor 2
Exception in thread "main" org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 30000
    at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1232)
    at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:156)
    at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:130)
    at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:75)
    at kafka.utils.ZkUtils$.apply(ZkUtils.scala:57)
    at kafka.admin.TopicCommand$.main(TopicCommand.scala:54)
    at kafka.admin.TopicCommand.main(TopicCommand.scala)

Any idea why this not work? It’s some simple test for zookeeper how check if works correctly? IP Addresses are from docker inspect

About this issue

Original URL
State: closed
Created 8 years ago
Reactions: 7
Comments: 16

Most upvoted comments

Most likely the broker’s advertised IP address is incorrect. I use an environment variable, DOCKER_HOST_IP, to use my Mac’s public IP address as the KAFKA_ADVERTISED_HOST_NAME value. Here are the relevant snippets from my scripts:

…

#
# Set the docker-compose.yml variable
#
if [[ -z "${DOCKER_HOST_IP-}" ]]; then
  docker_host_ip=$(docker run --rm --net host alpine ip address show eth0 | awk '$1=="inet" {print $2}' | cut -f1 -d'/')
  # Work around Docker for Mac 1.12.0-rc2-beta16 (build: 9493)
  if [[ $docker_host_ip = '192.168.65.2' ]]; then
    docker_host_ip=$(/sbin/ifconfig | grep -v '127.0.0.1' | awk '$1=="inet" {print $2}' | cut -f1 -d'/' | head -n 1)
  fi
  export DOCKER_HOST_IP=$docker_host_ip
fi

echo '==> building environment'

docker-compose build --pull

echo '==> launching environment'

docker-compose up -d

…

  kafka:
    image: wurstmeister/kafka:0.9.0.1
    links:
      - zookeeper:zk
    ports:
      - "9092:9092"
    environment:
      KAFKA_ADVERTISED_HOST_NAME: "${DOCKER_HOST_IP}"
      KAFKA_ZOOKEEPER_CONNECT: zk
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

…

jonbuffington on Aug 27, 2016

@jonbuffington @VatslavDS @mmullis

I find that nothing proposed here works.

My full docker-compose.yml:

version: '2'
services:
  zookeeper:
    image: wurstmeister/zookeeper
    ports:
      - "2181:2181"
      - "2888:2888"
      - "3888:3888"
  kafka:
    image: wurstmeister/kafka
    links:
      - zookeeper:zk
    ports:
      - "9092"
    environment:
      KAFKA_ADVERTISED_HOST_NAME: "${DOCKER_HOST_IP}"
      KAFKA_ZOOKEEPER_CONNECT: zk
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

What sense does it make to set KAFKA_ZOOKEEPER_CONNECT to a link alias, i.e. zk?

tscholak on Dec 19, 2016