You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Pablo Nebrera <pa...@eneotecnologia.com> on 2013/09/03 09:16:24 UTC
problems reasignating partitions
Hello
I have a cluster with:
* three nodes: rb01, rb02 and rb03
* three topics: rb_event, rb_monitor, rb_flow
* three partitions per topic
* replicas 2
The current status is:
Topic Partition Leader Replicas In-Sync replicas
-------------------------------------------------------------------------------
rb_event 0 2 2,1 2,0
1 2 1,2 2,0
2 2 1,2 2
-------------------------------------------------------------------------------
rb_flow 0 2 1,2 2,0
1 2 2,0 2,0
2 0 0,1 0,1
-------------------------------------------------------------------------------
rb_monitor 0 2 1,2 2
1 2 2,1 2,0
2 0 0,2 2,0
-------------------------------------------------------------------------------
I see something strange. For example:
* rb_event -> partition 0 has replicas 2,1 but In sync replicas are 2,0
-> that is normal ???
And the worst it when I try to reassign propertly the partitions:
The kafka-reassign-partitions.sh tries to write
to /admin/reassign_partitions the following:
{"partitions":[{"topic":"rb_event","partition":0,"replicas":[0,1]},{"topic":"rb_event","partition":1,"replicas":[1,2]},{"topic":"rb_event","partition":2,"replicas":[2,0]},{"topic":"rb_monitor","partition":0,"replicas":[2,0]},{"topic":"rb_monitor","partition":1,"replicas":[0,1]},{"topic":"rb_monitor","partition":2,"replicas":[1,2]}]}
When I access to zookeeper I do see:
[zk: rb02:2181,rb01:2181,rb03:2181(CONNECTED) 3] get
/admin/reassign_partitions
{ "partitions":[ { "partition":2, "replicas":[ 1, 2 ], "topic":"rb_monitor"
}, { "partition":0, "replicas":[ 2, 0 ], "topic":"rb_monitor" }, {
"partition":1, "replicas":[ 0, 1 ], "topic":"rb_monitor" }, {
"partition":0, "replicas":[ 0, 1 ], "topic":"rb_event" }, { "partition":2,
"replicas":[ 2, 0 ], "topic":"rb_event" } ], "version":1 }
And keep there for ever
What can I do ?
Thanks
Pablo
Re: problems reasignating partitions
Posted by Sriram Subramanian <sr...@linkedin.com>.
I have few questions before I can help you with your issue -
1. Which version/build of Kafka are you using?
2. Is the current status that you provided below before or after issuing
the reassign replica command?
3. If the answer to 2 above is "after issuing the command", how long was
it after which you checked zk state?
On 9/3/13 9:08 AM, "Jun Rao" <ju...@gmail.com> wrote:
>What you saw is not normal. In sync replicas should always be a subset of
>assigned replicas. Did that anomaly happen before or after reassigning
>partitions? Is that reproducible?
>
>Thanks,
>
>Jun
>
>
>
>On Tue, Sep 3, 2013 at 12:16 AM, Pablo Nebrera <
>pablonebrera@eneotecnologia.com> wrote:
>
>> Hello
>>
>> I have a cluster with:
>> * three nodes: rb01, rb02 and rb03
>> * three topics: rb_event, rb_monitor, rb_flow
>> * three partitions per topic
>> * replicas 2
>>
>>
>> The current status is:
>>
>> Topic Partition Leader Replicas In-Sync replicas
>>
>>
>>-------------------------------------------------------------------------
>>------
>> rb_event 0 2 2,1 2,0
>> 1 2 1,2 2,0
>> 2 2 1,2 2
>>
>>
>>-------------------------------------------------------------------------
>>------
>> rb_flow 0 2 1,2 2,0
>> 1 2 2,0 2,0
>> 2 0 0,1 0,1
>>
>>
>>-------------------------------------------------------------------------
>>------
>> rb_monitor 0 2 1,2 2
>> 1 2 2,1 2,0
>> 2 0 0,2 2,0
>>
>>
>>-------------------------------------------------------------------------
>>------
>>
>>
>> I see something strange. For example:
>>
>> * rb_event -> partition 0 has replicas 2,1 but In sync replicas are 2,0
>> -> that is normal ???
>>
>> And the worst it when I try to reassign propertly the partitions:
>>
>> The kafka-reassign-partitions.sh tries to write
>> to /admin/reassign_partitions the following:
>>
>>
>>
>>{"partitions":[{"topic":"rb_event","partition":0,"replicas":[0,1]},{"topi
>>c":"rb_event","partition":1,"replicas":[1,2]},{"topic":"rb_event","partit
>>ion":2,"replicas":[2,0]},{"topic":"rb_monitor","partition":0,"replicas":[
>>2,0]},{"topic":"rb_monitor","partition":1,"replicas":[0,1]},{"topic":"rb_
>>monitor","partition":2,"replicas":[1,2]}]}
>>
>>
>> When I access to zookeeper I do see:
>> [zk: rb02:2181,rb01:2181,rb03:2181(CONNECTED) 3] get
>> /admin/reassign_partitions
>> { "partitions":[ { "partition":2, "replicas":[ 1, 2 ],
>>"topic":"rb_monitor"
>> }, { "partition":0, "replicas":[ 2, 0 ], "topic":"rb_monitor" }, {
>> "partition":1, "replicas":[ 0, 1 ], "topic":"rb_monitor" }, {
>> "partition":0, "replicas":[ 0, 1 ], "topic":"rb_event" }, {
>>"partition":2,
>> "replicas":[ 2, 0 ], "topic":"rb_event" } ], "version":1 }
>>
>>
>> And keep there for ever
>>
>> What can I do ?
>>
>> Thanks
>>
>>
>>
>> Pablo
>>
Re: problems reasignating partitions
Posted by Jun Rao <ju...@gmail.com>.
What you saw is not normal. In sync replicas should always be a subset of
assigned replicas. Did that anomaly happen before or after reassigning
partitions? Is that reproducible?
Thanks,
Jun
On Tue, Sep 3, 2013 at 12:16 AM, Pablo Nebrera <
pablonebrera@eneotecnologia.com> wrote:
> Hello
>
> I have a cluster with:
> * three nodes: rb01, rb02 and rb03
> * three topics: rb_event, rb_monitor, rb_flow
> * three partitions per topic
> * replicas 2
>
>
> The current status is:
>
> Topic Partition Leader Replicas In-Sync replicas
>
> -------------------------------------------------------------------------------
> rb_event 0 2 2,1 2,0
> 1 2 1,2 2,0
> 2 2 1,2 2
>
> -------------------------------------------------------------------------------
> rb_flow 0 2 1,2 2,0
> 1 2 2,0 2,0
> 2 0 0,1 0,1
>
> -------------------------------------------------------------------------------
> rb_monitor 0 2 1,2 2
> 1 2 2,1 2,0
> 2 0 0,2 2,0
>
> -------------------------------------------------------------------------------
>
>
> I see something strange. For example:
>
> * rb_event -> partition 0 has replicas 2,1 but In sync replicas are 2,0
> -> that is normal ???
>
> And the worst it when I try to reassign propertly the partitions:
>
> The kafka-reassign-partitions.sh tries to write
> to /admin/reassign_partitions the following:
>
>
> {"partitions":[{"topic":"rb_event","partition":0,"replicas":[0,1]},{"topic":"rb_event","partition":1,"replicas":[1,2]},{"topic":"rb_event","partition":2,"replicas":[2,0]},{"topic":"rb_monitor","partition":0,"replicas":[2,0]},{"topic":"rb_monitor","partition":1,"replicas":[0,1]},{"topic":"rb_monitor","partition":2,"replicas":[1,2]}]}
>
>
> When I access to zookeeper I do see:
> [zk: rb02:2181,rb01:2181,rb03:2181(CONNECTED) 3] get
> /admin/reassign_partitions
> { "partitions":[ { "partition":2, "replicas":[ 1, 2 ], "topic":"rb_monitor"
> }, { "partition":0, "replicas":[ 2, 0 ], "topic":"rb_monitor" }, {
> "partition":1, "replicas":[ 0, 1 ], "topic":"rb_monitor" }, {
> "partition":0, "replicas":[ 0, 1 ], "topic":"rb_event" }, { "partition":2,
> "replicas":[ 2, 0 ], "topic":"rb_event" } ], "version":1 }
>
>
> And keep there for ever
>
> What can I do ?
>
> Thanks
>
>
>
> Pablo
>