Send messages to Kafka broker in the cloud from local machine

[Gurnur Wadhwani <gu...@gmail.com>]

Hello,

Can anyone suggest me a way out here ?

I am running a single Kafka broker in the cloud VM to which I want to send messages from my local machine.

The listeners config in my cloud machine looks like this:

listeners=INTERNAL://0.0.0.0:19092,EXTERNAL://0.0.0.0:9092
listener.security.protocol.map=INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT
advertised.listeners=INTERNAL://xxx:19092,EXTERNAL://xxx_external:9092
inter.broker.listener.name=INTERNAL

where, xxx is 127.0.0.1 and xxx_external is the public IP address of the machine.

When starting a producer and consumer in the cloud, on port 19092, Kafka is able to consume and produce messages.

But when starting a producer in my local machine using kafka binary with the following command:
bin/kafka-console-producer.sh --broker-list xxx_external:9092 --topic users 

I get the following error:
ERROR Error when sending message to topic users with key: null, value: 5 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Topic users not present in metadata after 60000 ms.

P.S.: The firewall on the port 9092 is opened, and when done a ‘tcpdump’ on the cloud machine with port 9092, I can see the packets arriving when a producer is started on local machine.

Thank you.

Best regards,
Gurnur