You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "James Cheng (JIRA)" <ji...@apache.org> on 2019/08/17 08:53:00 UTC
[jira] [Commented] (KAFKA-8808) Kafka Inconsistent Retention Period
across partitions
[ https://issues.apache.org/jira/browse/KAFKA-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16909626#comment-16909626 ]
James Cheng commented on KAFKA-8808:
------------------------------------
How did you create your topics? We ran into something like that recently. We were creating the topic with no config, and then setting the configuration immediately afterwards. In some cases (network congestion?) the broker would receive the configuration *before* the topic creation request. When that happened, the broker would ignore the config (because it was for a non-existent topic) and then would receive the topic creation request (which has no config, and therefore would use broker defaults)
> Kafka Inconsistent Retention Period across partitions
> -----------------------------------------------------
>
> Key: KAFKA-8808
> URL: https://issues.apache.org/jira/browse/KAFKA-8808
> Project: Kafka
> Issue Type: Bug
> Components: log, log cleaner
> Affects Versions: 1.0.0
> Reporter: Prashant
> Priority: Major
>
> Our topic is created with retention period of 3 days. Topic has four partitions. Broker level default is 12 hour.
> Some partition's segment get deleted even before 3 days. Logs show that segment is marked for deletion because it exceeded *43200000ms* = 12 hours.
>
> "INFO Found deletable segments with base offsets [43275] due to retention time *43200000ms* breach (kafka.log.Log)"
>
> This does not happen for all partitions. Post full cluster bounce , partitions facing this issue change.
>
> *Topic config :*
> Topic:TOPICNAME PartitionCount:4 ReplicationFactor:2 Configs:retention.ms=259200000,segment.ms=43200000
> Topic: TOPICNAME Partition: 0 Leader: 1 Replicas: 1,8 Isr: 1,8
> Topic: TOPICNAME Partition: 1 Leader: 2 Replicas: 2,9 Isr: 9,2
> Topic: TOPICNAME Partition: 2 Leader: 3 Replicas: 3,1 Isr: 1,3
> Topic: TOPICNAME Partition: 3 Leader: 4 Replicas: 4,2 Isr: 4,2
>
> *Broker config :*
> log.retention.ms=43200000
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)