You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Jun Rao (Jira)" <ji...@apache.org> on 2021/06/29 16:19:00 UTC
[jira] [Resolved] (KAFKA-12520) Producer state is needlessly
rebuilt on startup
[ https://issues.apache.org/jira/browse/KAFKA-12520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jun Rao resolved KAFKA-12520.
-----------------------------
Fix Version/s: 3.0.0
Assignee: Cong Ding (was: Dhruvil Shah)
Resolution: Fixed
merged the PR to trunk
> Producer state is needlessly rebuilt on startup
> -----------------------------------------------
>
> Key: KAFKA-12520
> URL: https://issues.apache.org/jira/browse/KAFKA-12520
> Project: Kafka
> Issue Type: Bug
> Reporter: Dhruvil Shah
> Assignee: Cong Ding
> Priority: Major
> Fix For: 3.0.0
>
>
> When we find a {{.swap}} file on startup, we typically want to rename and replace it as {{.log}}, {{.index}}, {{.timeindex}}, etc. as a way to complete any ongoing replace operations. These swap files are usually known to have been flushed to disk before the replace operation begins.
> One flaw in the current logic is that when we recover these swap files on startup, we end up truncating the producer state and rebuild it from scratch. This is unneeded as the replace operation does not mutate the producer state by itself. It is only meant to replace the {{.log}} file along with corresponding indices.
> Because of this unneeded producer state rebuild operation, we have seen multi-hour startup times for clusters that have large compacted topics.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)