[jira] [Created] (KAFKA-14075) Consumer Group deletion does not delete pending transactional offset commits

["Jeff Kim (Jira)" <ji...@apache.org>]

Jeff Kim created KAFKA-14075:
--------------------------------

             Summary: Consumer Group deletion does not delete pending transactional offset commits
                 Key: KAFKA-14075
                 URL: https://issues.apache.org/jira/browse/KAFKA-14075
             Project: Kafka
          Issue Type: Bug
            Reporter: Jeff Kim
            Assignee: Jeff Kim


In [GroupMetadata.removeAllOffsets()|https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/coordinator/group/GroupMetadata.scala#L729-L740] we pass in the offsets cache to delete pendingTransactionalOffsetCommits upon group deletion. So only transactional offset commits for topic partitions already in the offsets cache will be deleted.

However, we add a transactional offset commit to the offsets cache only after the commit/abort marker is written to the log in [GroupMetadata.completePendingTxnOffsetCommit()|https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/coordinator/group/GroupMetadata.scala#L692]

So even after a group deletion we can still have pending transactional offset commits for a group that's supposed to be deleted. The group metadata manager will throw an IllegalStateException [here|https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/coordinator/group/GroupMetadataManager.scala#L740] while loading group to memory. We will hit this exception on every load to group as long as the hanging transaction never completes. 

We should delete all pending transactional offset commits (instead of only topic partitions that exist in the offsets cache) when a group is deleted in GroupMetadata.removeOffsets()



--
This message was sent by Atlassian Jira
(v8.20.10#820010)