You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Khaireddine Rezgui (JIRA)" <ji...@apache.org> on 2018/04/12 10:28:00 UTC
[jira] [Comment Edited] (KAFKA-6535) Set default retention ms for
Streams repartition topics to Long.MAX_VALUE
[ https://issues.apache.org/jira/browse/KAFKA-6535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16435307#comment-16435307 ]
Khaireddine Rezgui edited comment on KAFKA-6535 at 4/12/18 10:27 AM:
---------------------------------------------------------------------
Hi guys, i updated the https://cwiki.apache.org/confluence/display/KAFKA/KIP-284%3A+Set+default+retention+ms+for+Streams+repartition+topics+to+Long.MAX_VALUE as accepted, can we proceed to the merge ?
was (Author: khairy):
Hi guys, i updated the [https://cwiki.apache.org/confluence/display/KAFKA/KIP-284%3A+Set+default+retention+ms+for+Streams+repartition+topics+to+Long.MAX_VALUE|KIP-284] as accepted, can we proceed to the merge ?
> Set default retention ms for Streams repartition topics to Long.MAX_VALUE
> -------------------------------------------------------------------------
>
> Key: KAFKA-6535
> URL: https://issues.apache.org/jira/browse/KAFKA-6535
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Guozhang Wang
> Assignee: Khaireddine Rezgui
> Priority: Major
> Labels: needs-kip, newbie
>
> After KIP-220 / KIP-204, repartition topics in Streams are transient, so it is better to set its default retention to infinity to allow any records be pushed to it with old timestamps (think: bootstrapping, re-processing) and just rely on the purging API to keeping its storage small.
> More specifically, in {{RepartitionTopicConfig}} we have a few default overrides for repartition topic configs, we should just add the override for {{TopicConfig.RETENTION_MS_CONFIG}} to set it to Long.MAX_VALUE. This still allows users to override themselves if they want via {{StreamsConfig.TOPIC_PREFIX}}. We need to add unit test to verify this update takes effect.
> In addition to the code change, we also need to have doc changes in streams/upgrade_guide.html specifying this default value change.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)