Kafka brokers listen on multiple ports

[Dan Gal <da...@gmail.com>]

Hello!

I have a few kafka clusters that communicate over 2 distinct ports (because
of a firewall between the newer clusters and the old ones). this is pretty
uncomfortable, so I would like to move the old default port to the new one
(that passes the firewall) through all my cluster.

All my clusters currently run kafka version 0.10.0.0.

For the deployment to pass silently, I thought to add the new port
alongside the old one in the older clusters for a transition period.

Is it possible to use 2 different ports for the same protocol (PLAINTEXT)
in the broker configuration? Can I simply put 2 connection strings in the
*listeners* config?

Thank you!
Dan

Re: Kafka brokers listen on multiple ports

[Kaufman Ng <ka...@confluent.io>]

Yes you can. Since you are gonna have 2 plaintext ports you would need to
name them differently. In the broker's configuration (server.properties)
you would need to specify listeners like this:

listeners=PLAINTEXT1://:9092,PLAINTEXT2://:9093

And map the "PLAINTEXT1" and "PLAINTEXT2" to the PLAINTEXT protocol:

listener.security.protocol.map=PLAINTEXT1:PLAINTEXT,PLAINTEXT2:PLAINTEXT

One additional config is you would have to specify the default listener
between brokers:

inter.broker.listener.name=PLAINTEXT1

Notice here I am using "PLAINTEXT1" rather than the default "PLAINTEXT"
listener name. If you keep the default "PLAINTEXT" listener name you would
NOT need to override inter.broker.listener.name like above.

More details are in this KIP:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-103%3A+Separation+of+Internal+and+External+traffic

Hope this helps.


On Fri, May 18, 2018 at 7:01 AM, Dan Gal <da...@gmail.com> wrote:

> Hello!
>
> I have a few kafka clusters that communicate over 2 distinct ports (because
> of a firewall between the newer clusters and the old ones). this is pretty
> uncomfortable, so I would like to move the old default port to the new one
> (that passes the firewall) through all my cluster.
>
> All my clusters currently run kafka version 0.10.0.0.
>
> For the deployment to pass silently, I thought to add the new port
> alongside the old one in the older clusters for a transition period.
>
> Is it possible to use 2 different ports for the same protocol (PLAINTEXT)
> in the broker configuration? Can I simply put 2 connection strings in the
> *listeners* config?
>
> Thank you!
> Dan
>



-- 
Kaufman Ng
+1 646 961 8063
Solutions Architect | Confluent | www.confluent.io