[GitHub] [kafka] vcrfxia opened a new pull request, #13566: KAFKA-14834: [10/N] Reserve repartition topic formats to include `isLatest`

["vcrfxia (via GitHub)" <gi...@apache.org>]

vcrfxia opened a new pull request, #13566:
URL: https://github.com/apache/kafka/pull/13566

   (This PR is stacked on https://github.com/apache/kafka/pull/13564. The first two commits do not need to be reviewed separately.)
   
   https://github.com/apache/kafka/pull/13564 introduced a new boolean `isLatest` into `Change` to indicate whether a change update represents the latest for the key. Even though `Change` is serialized into the table repartition topic, the new boolean does not need to be serialized in, because the table repartition map processor performs an optimization to drop records for which `isLatest = false`. If not for this optimization, the downstream table aggregate would have to drop such records instead, and `isLatest` would need to be serialized into the repartition topic. 
   
   In light of the possibility that `isLatest` may need to be serialized into the repartition topic in the future, e.g., if other downstream processors are added which need to distinguish between records for which `isLatest = true` vs `isLatest = false`, this PR reserves repartition topic formats which include `isLatest`. Reserving these formats now comes at no additional cost to users since a rolling bounce is already required for the upcoming release due to https://github.com/apache/kafka/pull/10747. If we don't reserve them now and instead have to add them later, then another bounce would be required at that time. Reserving formats is cheap, so we choose to do it now.
   
   This PR also pulls in the changes from https://github.com/apache/kafka/pull/13533, as one of the two would have to be rebased after the other is merged anyway.
   
   Co-authored-by: Matthias J. Sax <ma...@confluent.io>
   
   ### Committer Checklist (excluded from commit message)
   - [ ] Verify design and implementation 
   - [ ] Verify test coverage and CI build status
   - [ ] Verify documentation (including upgrade notes)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] mjsax merged pull request #13566: KAFKA-14834: [10/N] Reserve repartition topic formats to include `isLatest`

["mjsax (via GitHub)" <gi...@apache.org>]

mjsax merged PR #13566:
URL: https://github.com/apache/kafka/pull/13566


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org