[jira] [Updated] (KAFKA-21100) Defer materializing the record body until the record is confirmed to be retained

["Chia-Ping Tsai (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-21100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chia-Ping Tsai updated KAFKA-21100:
-----------------------------------
    Summary: Defer materializing the record body until the record is confirmed to be retained  (was: Defer to materialize the record body until it is confirmed to be retained)

> Defer materializing the record body until the record is confirmed to be retained
> --------------------------------------------------------------------------------
>
>                 Key: KAFKA-21100
>                 URL: https://issues.apache.org/jira/browse/KAFKA-21100
>             Project: Kafka
>          Issue Type: Improvement
>            Reporter: Chia-Ping Tsai
>            Assignee: majialong
>            Priority: Major
>
> This is similar to KAFKA-21082. 
> The other path, cleanInto / filterTo, could use a similar approach to defer loading the value until we know the record must be retained.
> The only fallback is when the oversized record is the newest value of its key and so must be kept. In this case, we could just keep the whole batch as-is (writeOriginalBatch already does this) to avoid reading the full value. The side effect is that batch is not compacted, but it won't violate the compaction guarantee, since its key is already in the offset map and older values in other batches are still cleaned. The oversized record will be dropped if it is not the newest one :)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)