Kafka client and ELB (round-robin DNS)

[BGCH <ig...@hivehome.com>]

Hi all,

briefly about the problem, if bootstrap.servers is a domain name and 
that domain name eventually starts pointing to different IP, kafka 
client will never try to re-resolve the new IP.

In our project we use java-based Kafka client 0.9. We configure 
bootstrap.servers  with the domain name, say kafka.ourvpc.io, which is 
managed by ELB (in our case it's just a round-robin DNS).
If we have two Kafka clusters and the first of them is killed after 
application is started, the application never reconnects to the second. 
Kafka client instantly tries to connect to the first IP despite the fact 
that DNS points to another IP.

Is this a know issue?  Are there any workarounds?

Thanks.


-- 
hivehome.com <http://www.hivehome.com>



Hive | London | Cambridge | Houston | Toronto
The information contained in or attached to this email is confidential and 
intended only for the use of the individual(s) to which it is addressed. It 
may contain information which is confidential and/or covered by legal 
professional or other privilege. The views expressed in this email are not 
necessarily the views of Centrica plc, and the company, its directors, 
officers or employees make no representation or accept any liability for 
their accuracy or completeness unless expressly stated to the contrary. 
Hive is the trading name of Centrica Connected Home Limited (company no: 
5782908), registered in England and Wales with its registered office at 
Millstream, Maidenhead Road, Windsor, Berkshire SL4 5GD.

Re: Kafka client and ELB (round-robin DNS)

[Stevo Slavić <ss...@gmail.com>]

See
http://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/java-dg-jvm-ttl.html

On Fri, Mar 3, 2017, 20:57 BGCH <ig...@hivehome.com> wrote:

> Hi all,
>
> briefly about the problem, if bootstrap.servers is a domain name and
> that domain name eventually starts pointing to different IP, kafka
> client will never try to re-resolve the new IP.
>
> In our project we use java-based Kafka client 0.9. We configure
> bootstrap.servers  with the domain name, say kafka.ourvpc.io, which is
> managed by ELB (in our case it's just a round-robin DNS).
> If we have two Kafka clusters and the first of them is killed after
> application is started, the application never reconnects to the second.
> Kafka client instantly tries to connect to the first IP despite the fact
> that DNS points to another IP.
>
> Is this a know issue?  Are there any workarounds?
>
> Thanks.
>
>
> --
> hivehome.com <http://www.hivehome.com>
>
>
>
> Hive | London | Cambridge | Houston | Toronto
> The information contained in or attached to this email is confidential and
> intended only for the use of the individual(s) to which it is addressed. It
> may contain information which is confidential and/or covered by legal
> professional or other privilege. The views expressed in this email are not
> necessarily the views of Centrica plc, and the company, its directors,
> officers or employees make no representation or accept any liability for
> their accuracy or completeness unless expressly stated to the contrary.
> Hive is the trading name of Centrica Connected Home Limited (company no:
> 5782908), registered in England and Wales with its registered office at
> Millstream, Maidenhead Road, Windsor, Berkshire SL4 5GD.
>