[jira] [Created] (KAFKA-21159) Clear next offsets to consume on task close to avoid stale commit after revive

["Bill Bejeck (Jira)" <ji...@apache.org>]

Bill Bejeck created KAFKA-21159:
-----------------------------------

             Summary:  Clear next offsets to consume on task close to avoid stale commit after revive
                 Key: KAFKA-21159
                 URL: https://issues.apache.org/jira/browse/KAFKA-21159
             Project: Kafka
          Issue Type: Bug
          Components: streams
            Reporter: Bill Bejeck
             Fix For: 4.5.0, 4.4.1


A revived active task (`closeDirtyAndRevive`) kept `nextOffsetsAndMetadataToBeConsumed`
from before the corruption. After the consumer is seeked back to its committed offset, a
commit driven by processor metadata (`needsCommit`, which commits all input partitions)
could fall back to that stale next offset and commit past records the task hasn't
reprocessed — skipping them if the task later moves, and purging them for a repartition
topic. The KAFKA-20416 `consumedOffsets.clear()` fix does not cover this path.

Fix: clear `nextOffsetsAndMetadataToBeConsumed` in `StreamTask.close()`, alongside
`consumedOffsets`, so a revived task has no stale fallback for a partition it hasn't
re-read.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)