Vfrdtyky

I'm trying to start a kafka service using docker-compose, and it should be able to be accessed inside and outside docker. So, it should be matter of setting the right advertisers inside and outside:

version: '3'

services:

  zookeeper:

    image: wurstmeister/zookeeper

    ports:

      - "2181:2181"

  kafka:

    image: wurstmeister/kafka

    ports:

      - "9094:9092"

    environment:

      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181

      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT

      KAFKA_LISTENERS: INSIDE://:9092,OUTSIDE://127.0.0.1:9094

      KAFKA_ADVERTISED_LISTENERS: INSIDE://:9092,OUTSIDE://127.0.0.1:9094

      KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE

The problem is that when I try to connect from outside the cluster, I don't get 127.0.0.1 as the name of the node, but the internal hostname:

$ kafkacat -L -b 127.0.0.1:9094

Metadata for all topics (from broker -1: 127.0.0.1:9092/bootstrap):

 1 brokers:

  broker 1001 at 91588ea968d4:9092

 28 topics:

 ...

Isn't the purpose of KAFKA_ADVERTISED_LISTENERS and KAFKA_LISTENERS to handle that situation? I tried setting KAFKA_ADVERTISED_HOST_NAME but it's ignored (one piece of documentation says it's deprecated, other one says that it's still active), but nevertheless that doesn't seem to be the answer, since I want two different advertised hostnames for two different networks.

I guess the old question remains: how to make kafka work inside and outside docker-compose?

Your config of the listeners looks correct, the problem is with your Docker Compose:

ports:

  - "9094:9092"

You’re mapping 9094 (‘Outside’) back to 9092 (‘Inside’) and thus when you connect you’re connecting to the ‘Inside’ listener. If you remove this line of config then your listener set up should work as intended.

For more info, see http://rmoff.net/2018/08/02/kafka-listeners-explained/