Kafka in Docker not working

My solution to this issue is slightly different. I configure Kafka to advertise on kafka host and, because it's exposed on the host machine on localhost:9092, I add an entry in /etc/hosts for kafka to resolve to localhost. By doing this Kafka can be accessed from both other Docker containers and from localhost.

docker-compose.yml:

  my-web-service:    build: ./my-web-service    ports:     - "8000:8000"    links:     - kafka  kafka:    image: "wurstmeister/kafka:0.10.2.0"    ports:     - "9092:9092"    hostname: kafka    links:      - zookeeper    environment:     - KAFKA_ADVERTISED_HOST_NAME=kafka     - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181     - KAFKA_ADVERTISED_PORT=9092  zookeeper:    image: wurstmeister/zookeeper    ports:      - "2181:2181"

Updated hosts file:

more /etc/hosts127.0.0.1       localhost kafka

For developing app in localhost, there is a solution in the documentation: "HOSTNAME_COMMAND"

kafka:  image: wurstmeister/kafka  ports:    - 9092:9092environment:  KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181  HOSTNAME_COMMAND: "route -n | awk '/UG[ \t]/{print $$2}'"

Hope this help others...

here's an improved version of @radek1st answer.

links is the old docker way, networks is the current method.

imo, making any sort of system change isn't good and should never be needed. it also kind of defeats the purpose of using Docker.

version: '2.1'networks:  sb:    driver: bridgeservices:  zookeeper:    image: confluentinc/cp-zookeeper:latest    container_name: zookeeper    hostname: zookeeper    networks:     - sb    ports:      - "2181:2181"    environment:      ZOOKEEPER_CLIENT_PORT: 2181      ZOOKEEPER_TICK_TIME: 2000  kafka:    image: confluentinc/cp-kafka:latest    container_name: kafka    hostname: ${KAFKA_HOSTNAME:-kafka}    depends_on:      - zookeeper    networks:     - sb    ports:      - "9092:9092"    environment:      KAFKA_BROKER_ID: 1      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181      KAFKA_ADVERTISED_HOST_NAME: ${KAFKA_HOSTNAME:-kafka}      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://${KAFKA_HOSTNAME:-kafka}:9092      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1

Then I use the follow bash script to start things. This allows me to override the Kafka hostname for local development. ./startup.sh localhost

#!/bin/bashecho KAFKA_HOSTNAME=${1:-kafka} > .envdocker-compose up -d

Read more -

• https://docs.docker.com/compose/networking
• https://docs.docker.com/compose/environment-variables/
• https://docs.docker.com/compose/compose-file/#variable-substitution