You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Manikumar (JIRA)" <ji...@apache.org> on 2018/05/23 17:04:00 UTC
[jira] [Resolved] (KAFKA-6356) UnknownTopicOrPartitionException &
NotLeaderForPartitionException and log deletion happening with retention
bytes kept at -1.
[ https://issues.apache.org/jira/browse/KAFKA-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manikumar resolved KAFKA-6356.
------------------------------
Resolution: Not A Problem
old data is discarded after log retention period or when the log reaches retention time. In this case, you may need to increase retention period. replication errors looks normal. Please reopen if you think the issue still exists
Post these kind of queries to users@kafka.apache.org mailing list (http://kafka.apache.org/contact) for quicker response.
> UnknownTopicOrPartitionException & NotLeaderForPartitionException and log deletion happening with retention bytes kept at -1.
> -----------------------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-6356
> URL: https://issues.apache.org/jira/browse/KAFKA-6356
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.10.1.0
> Environment: Cent OS 7.2,
> HDD : 2Tb,
> CPUs: 56 cores,
> RAM : 256GB
> Reporter: kaushik srinivas
> Priority: Major
> Attachments: configs.txt, stderr_b0, stderr_b1, stderr_b2, stdout_b0, stdout_b1, stdout_b2, topic_description, topic_offsets
>
>
> Facing issues in kafka topic with partitions and replication factor of 3.
> Config used :
> No of partitions : 20
> replication factor : 3
> No of brokers : 3
> Memory for broker : 32GB
> Heap for broker : 12GB
> Producer is run to produce data for 20 partitions of a single topic.
> But observed that partitions for which the leader is one of the broker(broker-1), the offsets are never incremented and also we see log file with 0MB size in the broker disk.
> Seeing below error in the brokers :
> error 1:
> 2017-12-13 07:11:11,191] ERROR [ReplicaFetcherThread-0-2], Error for partition [test2,5] to broker 2:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. (kafka.server.ReplicaFetcherThread)
> error 2:
> [2017-12-11 12:19:41,599] ERROR [ReplicaFetcherThread-0-2], Error for partition [test1,13] to broker 2:org.apache.kafka.common.errors.NotLeaderForPartitionException: This server is not the leader for that topic-partition. (kafka.server.ReplicaFetcherThread)
> Attaching,
> 1. error and std out files of all the brokers.
> 2. kafka config used.
> 3. offsets and topic description.
> Retention bytes was kept to -1 and retention period 96 hours.
> But still observing some of the log files deleting at the broker,
> from logs :
> [2017-12-11 12:20:20,586] INFO Deleting index /var/lib/mesos/slave/slaves/7b319cf4-f06e-4a35-a6fe-fd4fcc0548e6-S7/frameworks/7b319cf4-f06e-4a35-a6fe-fd4fcc0548e6-0006/executors/ckafka__5f085d0c-e296-40f0-a686-8953dd14e4c6/runs/506a1ce7-23d1-45ea-bb7c-84e015405285/kafka-broker-data/broker-1/test1-12/00000000000000000000.timeindex (kafka.log.TimeIndex)
> [2017-12-11 12:20:20,587] INFO Deleted log for partition [test1,12] in /var/lib/mesos/slave/slaves/7b319cf4-f06e-4a35-a6fe-fd4fcc0548e6-S7/frameworks/7b319cf4-f06e-4a35-a6fe-fd4fcc0548e6-0006/executors/ckafka__5f085d0c-e296-40f0-a686-8953dd14e4c6/runs/506a1ce7-23d1-45ea-bb7c-84e015405285/kafka-broker-data/broker-1/test1-12. (kafka.log.LogManager)
> We are expecting the logs to be never delete if retention bytes set to -1.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)