Re: enforceRebalance using kafka admin APIs anf not consumer client API.

[John Roesler <vv...@apache.org>]

Hi Mazen,

This sounds like a good use case. If you’d like, you can start a KIP to add an enforceRebalance() method to the admin client interface. Feel free to ask here for any guidance on the KIP process itself.

Regarding your usage, you will actually have to call poll(). That is the point at which broker communication actually takes place. 

I hope this helps,
John

On Mon, Mar 29, 2021, at 01:44, Mazen Ezzeddine wrote:
> Dear all,
> I am using Kafka admin client where I am performing some computations 
> on the offsets (committed, last, …) to derive per-partition and 
> per-consumer information (on a certain consumer group) such as arrival 
> rate, lag, consumption rate etc… when certain conditions are met I want 
> to enforce rebalance on the consumer group being monitored.
> The issue is that the enforceRebalance is a consumer client API (and 
> not admin client API), so could you please provide information on the 
> most seamless way to enforce consumer group scenario in a situation 
> like the above.
> 
> Note that I tried workarounds like dynamically adding a new consumer 
> through Kubernetes client API  (as my cluster and clients are running 
> on kubernetes) to trigger a rebalance but preferably I don’t want to 
> use this workaround.
> 
> If I added to my admin client class a kafka consumer to be a member of 
> the consumer group without polling (calling poll) just for the purpose 
> of triggering a rebalance using the consumer API would that work. 
> Indeed I tried this scenario and it looks like no compilation or 
> runtime errors but I am not seeing in the logs that a group rebalance 
> has taken place ? am I missing something? any hint please?
> 
> Thank you.
> 
>