Kafka running in VPN

[Subroto Sanyal <sa...@gmail.com>]

Hello

I am using Kafka-0.10.0
My Kafka brokers are running on a 3 node cluster in a VPN. The consumer and
producer are not part of VPN; so I use ssh tunnels and update the bootstrap
servers on the Kafka Client (consumer and producer) config accordingly.
The Kafka Client is able to initiate the primary connection and I see a
topic is also getting created successfully but, further communication is
not happening.
With little debugging found out that there is something like Metadata
Updater which gets the broker advertised ports and further communication
using the same InsetSocketAddress.

Is there any way to let the Kafka Clients not use/update the broker address?

-- 
Cheers,
*Subroto Sanyal*

Re: Kafka running in VPN

[Ben Stopford <be...@confluent.io>]

The bootstrap servers are only used to make an initial connection. From
there the clients's request metadata which provides them with a 'map' of
the cluster. The addresses in the metadata are those registered in
Zookeeper by each broker. They don't relate to the bootstrap list in any
way. You can change these from the default addresses the brokers bind to
via the advertised.listeners / listeners broker properies.

-B

On Wed, Mar 22, 2017 at 1:28 PM Subroto Sanyal <sa...@gmail.com>
wrote:

> Hello
>
> I am using Kafka-0.10.0
> My Kafka brokers are running on a 3 node cluster in a VPN. The consumer and
> producer are not part of VPN; so I use ssh tunnels and update the bootstrap
> servers on the Kafka Client (consumer and producer) config accordingly.
> The Kafka Client is able to initiate the primary connection and I see a
> topic is also getting created successfully but, further communication is
> not happening.
> With little debugging found out that there is something like Metadata
> Updater which gets the broker advertised ports and further communication
> using the same InsetSocketAddress.
>
> Is there any way to let the Kafka Clients not use/update the broker
> address?
>
> --
> Cheers,
> *Subroto Sanyal*
>