You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Gabriel Giussi <ga...@gmail.com> on 2022/05/26 19:57:52 UTC

Can CooperativeStickyAssignor be used with transactions?

This is the scenario I have in mind

1. Client A gets assigned partitions P1 and P2.
2. Client A polls a message with offset X from P1, opens a transaction and
produces to some output topic.
3. Client B joins the group and gets assigned P2
4. Client A tries to sendOffsets with group metadata but is fenced (due to
the rebalance that bumped the generation.id). At this moment I can't use
that producer anymore so I will just "ignore" that message, since the
broker will abort the transaction.
5. Client A polls again and will get the message with offset X + 1, since
that partition wasn't revoked due to the cooperative rebalance.

In an eager rebalance this can't happen since the rebalance that bumped the
generation.id and fenced my producer will also revoke the partitions and
force my consumer to fetch the offset again and poll the message with
offset X again.
I couldn't test this locally yet since isn't trivial, but could this happen
in theory or am I missing something?

Thanks.