You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by "Yu, Libo " <li...@citi.com> on 2013/11/28 21:28:09 UTC

addition of a partition

Hi team,

I am reading this link:
https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-5.AddPartitionTool
and this JIRA https://issues.apache.org/jira/i#browse/KAFKA-1030. I have a couple of questions.

After adding a partition by using the tool, should the consumer processes be restarted? If auto.offset.reset
is set to the largest for my consumer processes, will be it changed during partition addition? And is there
chance of message loss? Thanks.



Regards,

Libo

RE: addition of a partition

Posted by "Yu, Libo " <li...@citi.com>.

Thanks, Jun. It is good to know that. Currently, auto.offset.reset  is set to largest for all our consumers.
It seems we have to change it to smallest.

Regards,

Libo

-----Original Message-----
From: Jun Rao [mailto:junrao@gmail.com] 
Sent: Friday, November 29, 2013 1:17 AM
To: users@kafka.apache.org
Subject: Re: addition of a partition

The consumer should be able to pick up the new partitions automatically. If is set to the largest, the consumer may miss some data in the new partitions. To avoid data loss, you have to set auto.offset.reset to smallest.

Thanks,

Jun

On Thu, Nov 28, 2013 at 12:28 PM, Yu, Libo <li...@citi.com> wrote:

> Hi team,
>
> I am reading this link:
>
> https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Re
> plicationtools-5.AddPartitionTool and this JIRA 
> https://issues.apache.org/jira/i#browse/KAFKA-1030. I have a couple of 
> questions.
>
> After adding a partition by using the tool, should the consumer 
> processes be restarted? If auto.offset.reset is set to the largest for 
> my consumer processes, will be it changed during partition addition? 
> And is there chance of message loss? Thanks.
>
>
>
> Regards,
>
> Libo
>
>

Re: addition of a partition

Posted by Jun Rao <ju...@gmail.com>.

The consumer should be able to pick up the new partitions automatically. If
auto.offset.reset is set to the largest, the consumer may miss some data in
the new partitions. To avoid data loss, you have to set auto.offset.reset
to smallest.

Thanks,

Jun

On Thu, Nov 28, 2013 at 12:28 PM, Yu, Libo <li...@citi.com> wrote:

> Hi team,
>
> I am reading this link:
>
> https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-5.AddPartitionTool
> and this JIRA https://issues.apache.org/jira/i#browse/KAFKA-1030. I have
> a couple of questions.
>
> After adding a partition by using the tool, should the consumer processes
> be restarted? If auto.offset.reset
> is set to the largest for my consumer processes, will be it changed during
> partition addition? And is there
> chance of message loss? Thanks.
>
>
>
> Regards,
>
> Libo
>
>