Java client 2.4.1 doesn't always handle IP changes

[Michał Łowicki <ml...@gmail.com>]

Hey,

Had incident where one broker died and got later different IP address. Some
clients / pods (everything lives on K8s) detected IP change and logged:

[Producer clientId=producer-1] Hostname for node 10000 changed from XXXX to
> YYYY.


(logged by org.apache.kafka.clients.ClusterConnectionStates) but many
didn't and they couldn't actually send anything, continuously timing out.
Any clue if there were some know issues with that and maybe something has
been fixed in > 2.4.1?

In the meantime I'm also checking DNS caching but nothing at OS-level but
still verifying JVM (I'm using Java 14).

-- 
BR,
Michał Łowicki

Re: Java client 2.4.1 doesn't always handle IP changes

[Micha�� ��owicki <ml...@gmail.com>]

Checked Java DNS resolution caching:

```
sun.net.InetAddressCachePolicy.get();
sun.net.InetAddressCachePolicy.getNegative();
```

and those return 30 and 10 respectively. So it seems fine and it shouldn't cache for too long.

On 2021/05/14 12:55:00, Michał Łowicki <ml...@gmail.com> wrote: 
> Hey,
> 
> Had incident where one broker died and got later different IP address. Some
> clients / pods (everything lives on K8s) detected IP change and logged:
> 
> [Producer clientId=producer-1] Hostname for node 10000 changed from XXXX to
> > YYYY.
> 
> 
> (logged by org.apache.kafka.clients.ClusterConnectionStates) but many
> didn't and they couldn't actually send anything, continuously timing out.
> Any clue if there were some know issues with that and maybe something has
> been fixed in > 2.4.1?
> 
> In the meantime I'm also checking DNS caching but nothing at OS-level but
> still verifying JVM (I'm using Java 14).
> 
> -- 
> BR,
> Michał Łowicki
>