[jira] [Created] (KAFKA-21101) Drop the remaining records of a batch when encountering an invalid record

["Chia-Ping Tsai (Jira)" <ji...@apache.org>]

Chia-Ping Tsai created KAFKA-21101:
--------------------------------------

             Summary: Drop the remaining records of a batch when encountering an invalid record
                 Key: KAFKA-21101
                 URL: https://issues.apache.org/jira/browse/KAFKA-21101
             Project: Kafka
          Issue Type: Improvement
            Reporter: Chia-Ping Tsai
            Assignee: Chia-Ping Tsai


Currently, an invalid record marks the partition "uncleanable", which blocks the cleaner from compacting all other records in that partition. Since the record is already invalid regardless of the root cause, nobody can read it or anything after it in the same batch (consumers parse a batch sequentially and fail at the same point), so dropping them loses nothing observable. We should offer an option to drop the invalid record and the remaining records of its batch, and keep the partition cleanable. Records before the invalid one are parsed successfully and are retained as usual.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)