You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by "Yu, Libo " <li...@citi.com> on 2014/01/14 21:38:51 UTC

purging a topic

Hi folks,

I am writing a tool to "purge" the pending topics for a user. Assume the user has never
consumed this topic previously. If I create all the nodes on the path
/consumers/[myuser]/offsets/[mytopic]/[partition] and put the maximum
available offset to the node, is that enough to let the consumer ignore the current pending
messages?

Regards,

Libo

RE: purging a topic

Posted by "Yu, Libo " <li...@citi.com>.

Thanks, Jun. I checked ImportZkOffsets. It is not very handy and cannot automate
all steps for purging purpose.

Regards,

Libo

-----Original Message-----
From: Jun Rao [mailto:junrao@gmail.com] 
Sent: Tuesday, January 14, 2014 11:43 PM
To: users@kafka.apache.org
Subject: Re: purging a topic

You would need to first stop the consumer, update the offset in ZK and then restart the consumer. Also, have you looked at the tool ImportZkOffsets?

Thanks,

Jun

On Tue, Jan 14, 2014 at 12:38 PM, Yu, Libo <li...@citi.com> wrote:

> Hi folks,
>
> I am writing a tool to "purge" the pending topics for a user. Assume 
> the user has never consumed this topic previously. If I create all the 
> nodes on the path /consumers/[myuser]/offsets/[mytopic]/[partition] 
> and put the maximum available offset to the node, is that enough to 
> let the consumer ignore the current pending messages?
>
> Regards,
>
> Libo
>
>

Re: purging a topic

Posted by Jun Rao <ju...@gmail.com>.

You would need to first stop the consumer, update the offset in ZK and then
restart the consumer. Also, have you looked at the tool ImportZkOffsets?

Thanks,

Jun

On Tue, Jan 14, 2014 at 12:38 PM, Yu, Libo <li...@citi.com> wrote:

> Hi folks,
>
> I am writing a tool to "purge" the pending topics for a user. Assume the
> user has never
> consumed this topic previously. If I create all the nodes on the path
> /consumers/[myuser]/offsets/[mytopic]/[partition] and put the maximum
> available offset to the node, is that enough to let the consumer ignore
> the current pending
> messages?
>
> Regards,
>
> Libo
>
>

Re: purging a topic

Posted by Guozhang Wang <wa...@gmail.com>.

Hello Libo,

This should work, but I am more curious about your exact usage scenarios.
Would setting the default offset reset to largest be appropriate for you?

Guozhang

On Tue, Jan 14, 2014 at 12:38 PM, Yu, Libo <li...@citi.com> wrote:

> Hi folks,
>
> I am writing a tool to "purge" the pending topics for a user. Assume the
> user has never
> consumed this topic previously. If I create all the nodes on the path
> /consumers/[myuser]/offsets/[mytopic]/[partition] and put the maximum
> available offset to the node, is that enough to let the consumer ignore
> the current pending
> messages?
>
> Regards,
>
> Libo
>
>

-- 
-- Guozhang