You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2019/02/05 01:48:00 UTC
[jira] [Commented] (KAFKA-7897) Invalid use of epoch cache
following message format downgrade
[ https://issues.apache.org/jira/browse/KAFKA-7897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16760382#comment-16760382 ]
ASF GitHub Bot commented on KAFKA-7897:
---------------------------------------
hachikuji commented on pull request #6232: KAFKA-7897; Clear leader epoch cache after message format downgrade
URL: https://github.com/apache/kafka/pull/6232
If the message format is downgraded, we should clear the leader epoch cache so that it is not mistakenly used for truncation. We want to revert to truncation by high watermark.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> Invalid use of epoch cache following message format downgrade
> -------------------------------------------------------------
>
> Key: KAFKA-7897
> URL: https://issues.apache.org/jira/browse/KAFKA-7897
> Project: Kafka
> Issue Type: Bug
> Reporter: Jason Gustafson
> Assignee: Jason Gustafson
> Priority: Major
>
> Message format downgrades are not supported, but they generally work as long as broker/clients at least can continue to parse both message formats. After a downgrade, the truncation logic should revert to using the high watermark, but currently we use the existence of any cached epoch as the sole prerequisite in order to leverage OffsetsForLeaderEpoch. This has the effect of causing a massive truncation after startup which causes re-replication.
> I think our options to fix this are to either 1) clear the cache when we notice a downgrade, or 2) forbid downgrades and raise an error.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)