You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by florent valdelievre <fl...@gmail.com> on 2014/09/30 12:03:50 UTC
After creating a topic, broker gets dropped from ISR
Hi again,
192.168.1.180
Zk: 192.168.1.180:2181
Kafka: 9092
Broker.id = 1
zookeeper.connect=192.168.1.180:2181
------
192.168.1.190
Zk: 192.168.1.180:2181
Kafka: 9092
Broker.id = 2
zookeeper.connect=192.168.1.180:2181
------
I start both Kafka-server
I create a topic using the following command ( the command is launch on
192.168.1.180)
kafka-topics.sh --create --zookeeper 192.168.1.180:2181 --topic
hibe-test-12 --partitions 1 --replication-factor 2
*Stdout on broker.id <http://broker.id>=1*
[2014-09-30 05:51:01,586] INFO Created log for partition [hibe-test-12,0]
in /home/shopmedia/apps/kafka/log with properties {segment.index.bytes ->
10485760, file.delete.delay.ms -> 60000, segment.bytes -> 536870912,
flush.ms -> 9223372036854775807, delete.retention.ms -> 86400000,
index.interval.bytes -> 4096, retention.bytes -> -1, cleanup.policy ->
delete, segment.ms -> 604800000, max.message.bytes -> 1000012,
flush.messages -> 9223372036854775807, min.cleanable.dirty.ratio -> 0.5,
retention.ms -> 3600000}. (kafka.log.LogManager)
[2014-09-30 05:51:01,588] WARN Partition [hibe-test-12,0] on broker 1: No
checkpointed highwatermark is found for partition [hibe-test-12,0]
(kafka.cluster.Partition)
[2014-09-30 05:51:19,366] INFO Partition [hibe-test-12,0] on broker 1:
Shrinking ISR for partition [hibe-test-12,0] from 1,2 to 1
(kafka.cluster.Partition)
------
*Stdout on broker.id <http://broker.id>=2*
[2014-09-30 05:51:11,952] ERROR [ReplicaFetcherThread-0-1], Error for
partition [hibe-test-12,0] to broker 1:class kafka.common.UnknownException
(kafka.server.ReplicaFetcherThread)
[2014-09-30 05:51:11,954] ERROR [KafkaApi-2] error when handling request
Name: FetchRequest; Version: 0; CorrelationId: 2647; ClientId:
ReplicaFetcherThread-0-1; ReplicaId: 2; MaxWait: 500 ms; MinBytes: 1 bytes;
RequestInfo: [hibe-test-12,0] -> PartitionFetchInfo(0,1048576)
(kafka.server.KafkaApis)
kafka.common.KafkaException: Shouldn't set logEndOffset for replica 2
partition [hibe-test-12,0] since it's local
This error is high CPU usage, roughly 40% CPU usage until i kill
kafka-server
Please note that i have removed log fdata rom both server beforehand as
well as remove zk data using : zkCli.sh
rmr /brokers
Broker.id = 2 never gets in the ISR
Do you have an idea what is causing this error ?
Re: After creating a topic, broker gets dropped from ISR
Posted by Jun Rao <ju...@gmail.com>.
Could you check that the broker host registered in ZK is the ip that you
are expecting?
Thanks,
Jun
On Tue, Sep 30, 2014 at 3:03 AM, florent valdelievre <
florentvaldelievre@gmail.com> wrote:
> Hi again,
>
> 192.168.1.180
> Zk: 192.168.1.180:2181
> Kafka: 9092
> Broker.id = 1
> zookeeper.connect=192.168.1.180:2181
>
> ------
>
> 192.168.1.190
> Zk: 192.168.1.180:2181
> Kafka: 9092
> Broker.id = 2
> zookeeper.connect=192.168.1.180:2181
>
> ------
>
> I start both Kafka-server
> I create a topic using the following command ( the command is launch on
> 192.168.1.180)
>
> kafka-topics.sh --create --zookeeper 192.168.1.180:2181 --topic
> hibe-test-12 --partitions 1 --replication-factor 2
>
> *Stdout on broker.id <http://broker.id>=1*
> [2014-09-30 05:51:01,586] INFO Created log for partition [hibe-test-12,0]
> in /home/shopmedia/apps/kafka/log with properties {segment.index.bytes ->
> 10485760, file.delete.delay.ms -> 60000, segment.bytes -> 536870912,
> flush.ms -> 9223372036854775807, delete.retention.ms -> 86400000,
> index.interval.bytes -> 4096, retention.bytes -> -1, cleanup.policy ->
> delete, segment.ms -> 604800000, max.message.bytes -> 1000012,
> flush.messages -> 9223372036854775807, min.cleanable.dirty.ratio -> 0.5,
> retention.ms -> 3600000}. (kafka.log.LogManager)
> [2014-09-30 05:51:01,588] WARN Partition [hibe-test-12,0] on broker 1: No
> checkpointed highwatermark is found for partition [hibe-test-12,0]
> (kafka.cluster.Partition)
> [2014-09-30 05:51:19,366] INFO Partition [hibe-test-12,0] on broker 1:
> Shrinking ISR for partition [hibe-test-12,0] from 1,2 to 1
> (kafka.cluster.Partition)
>
> ------
>
>
> *Stdout on broker.id <http://broker.id>=2*
>
> [2014-09-30 05:51:11,952] ERROR [ReplicaFetcherThread-0-1], Error for
> partition [hibe-test-12,0] to broker 1:class kafka.common.UnknownException
> (kafka.server.ReplicaFetcherThread)
> [2014-09-30 05:51:11,954] ERROR [KafkaApi-2] error when handling request
> Name: FetchRequest; Version: 0; CorrelationId: 2647; ClientId:
> ReplicaFetcherThread-0-1; ReplicaId: 2; MaxWait: 500 ms; MinBytes: 1 bytes;
> RequestInfo: [hibe-test-12,0] -> PartitionFetchInfo(0,1048576)
> (kafka.server.KafkaApis)
> kafka.common.KafkaException: Shouldn't set logEndOffset for replica 2
> partition [hibe-test-12,0] since it's local
>
> This error is high CPU usage, roughly 40% CPU usage until i kill
> kafka-server
>
> Please note that i have removed log fdata rom both server beforehand as
> well as remove zk data using : zkCli.sh
> rmr /brokers
>
> Broker.id = 2 never gets in the ISR
>
> Do you have an idea what is causing this error ?
>