How to decrease number of replicas

[Guangle Fan <fa...@gmail.com>]

Hi, Folks

We have an online kafka cluster v0.8.1.1.
After running a partition reassignment script which maps each partition to
3 replicas. But growth of data is out of my expectation, and I really need
to decrease replicas for each partition to 2 or 1.

What's the best way to do this ?

Thanks !

GFan

Re: How to decrease number of replicas

[Guangle Fan <fa...@gmail.com>]

Well I solved my problem. For record, the problem was prior reassignment
was not finished while new reassignment was kicked out, in which case
replicas list won't change as expected. Given our data is at relatively
large scale, reassignment just stayed without progress for one day.
The Solution is rolling restart stuck leaders for each partition, and sent
out replica reassignment once again.

On Thu, Jun 18, 2015 at 5:37 PM, Guangle Fan <fa...@gmail.com> wrote:

> If I use the same approach to reassign smaller number of replicas to the
> same partition, I got this error :
>
> (0,5,1,6,2,3) are the current replica, and (6) is the new list I want to
> assign to topic partition 0
>
> Assigned replicas (0,5,1,6,2,3) don't match the list of replicas for
> reassignment (6) for partition [topic,0]
>
> On Thu, Jun 18, 2015 at 5:03 PM, Guangle Fan <fa...@gmail.com> wrote:
>
>> Hi, Folks
>>
>> We have an online kafka cluster v0.8.1.1.
>> After running a partition reassignment script which maps each partition
>> to 3 replicas. But growth of data is out of my expectation, and I really
>> need to decrease replicas for each partition to 2 or 1.
>>
>> What's the best way to do this ?
>>
>> Thanks !
>>
>> GFan
>>
>
>

Re: How to decrease number of replicas

[Guangle Fan <fa...@gmail.com>]

If I use the same approach to reassign smaller number of replicas to the
same partition, I got this error :

(0,5,1,6,2,3) are the current replica, and (6) is the new list I want to
assign to topic partition 0

Assigned replicas (0,5,1,6,2,3) don't match the list of replicas for
reassignment (6) for partition [topic,0]

On Thu, Jun 18, 2015 at 5:03 PM, Guangle Fan <fa...@gmail.com> wrote:

> Hi, Folks
>
> We have an online kafka cluster v0.8.1.1.
> After running a partition reassignment script which maps each partition to
> 3 replicas. But growth of data is out of my expectation, and I really need
> to decrease replicas for each partition to 2 or 1.
>
> What's the best way to do this ?
>
> Thanks !
>
> GFan
>