Re: Consumer Group got stuck in CompletingRebalance

[Karolis Pocius <ka...@sentiance.com.INVALID>]

Don't mean to resurrect an old thread, but in case you haven't found the
answer, this sounds exactly like the issue I've encountered after upgrading
to 2.4.0

It was caused by this https://issues.apache.org/jira/browse/KAFKA-9752

On Tue, Apr 14, 2020 at 10:08 AM James Brown <jb...@easypost.com> wrote:

> We had a consumer crash and restart itself a bunch of times, and then the
> group got stuck in state CompletingRebalance. All of the consumers were
> dead (I checked that there are no processes running on the host in the
> output of kafka-consumer-groups.sh --describe --group group-name
> --members), but the group was still rebalancing. If I tried to start up a
> consumer, it just timed out trying to join the group.
>
> I tried to delete the group, but that fails with
>
> * Group 'trackers-etl' could not be deleted due to:
> java.util.concurrent.ExecutionException:
> org.apache.kafka.common.errors.GroupNotEmptyException: The group is not
> empty.
>
> I ended up restarting the broker that was the leader for that group; that
> fixed the issue but is obviously a pretty heavy-weight solution.
>
> There was nothing interesting in the logs on that broker; the last thing it
> logged about that group was:
>
> [2020-04-13 21:37:20,308] INFO [GroupCoordinator 6]: Stabilized group
> trackers-etl generation 4987 (__consumer_offsets-29)
> (kafka.coordinator.group.GroupCoordinator)
>
> Has anyone seen this before? Should I file a JIRA ticket? Was there a
> better process than restarting the broker?
> --
> James Brown
> Systems Engineer
>