[jira] [Created] (KAFKA-10324) Pre-0.11 consumers can get stuck when messages are downconverted from V2 format

["Tommy Becker (Jira)" <ji...@apache.org>]

Tommy Becker created KAFKA-10324:
------------------------------------

             Summary: Pre-0.11 consumers can get stuck when messages are downconverted from V2 format
                 Key: KAFKA-10324
                 URL: https://issues.apache.org/jira/browse/KAFKA-10324
             Project: Kafka
          Issue Type: Bug
            Reporter: Tommy Becker


As noted in KAFKA-5443, The V2 message format preserves a batch's lastOffset even if that offset gets removed due to log compaction. If a pre-0.11 consumer seeks to such an offset and issues a fetch, it will get an empty batch, since offsets prior to the requested one are filtered out during down-conversion. KAFKA-5443 added consumer-side logic to advance the fetch offset in this case, but this leaves old consumers unable to consume these topics.

The exact behavior varies depending on consumer version. The 0.10.0.0 consumer throws RecordTooLargeException and dies, believing that the record must not have been returned because it was too large. The 0.10.1.0 consumer simply spins fetching the same empty batch over and over.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)