You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Casey Daniell <ca...@hotmail.com> on 2015/10/27 16:09:47 UTC
Drop Topic from Consumer
We are using Kafka 0.8.1 and have a Consumer that used to pull multiple topics. One of these topics is no longer needed by this consumer/group, so when I look at
/usr/local/kafka/bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --zkconnect localhost --group <ServerName>
We see the LAG for this measurement is high, but we know why. Is there anyway to let ZK know that this consumer group will never catch up for this topic, since it's not consumed any longer?
Using kafka brokers to obtain topic metadata vs zookeepers
Posted by Virendra Pratap Singh <vp...@yahoo-inc.com.INVALID>.
The topic level metadata is available with every broker in kafka and also in zookeepers. Kafka producer API uses list of brokers while the high level kafka consumer (scala) uses the zookeeper host list.
Is there a reason for this two different behavior. And would it suffice to use the zookeeper to get topic and related partition and leader broker info?
Guess I am treating zookeeper quorum as a VIP alternative to obtain topic/partition/broker info and not rely on setting up a broker VIP or a list of brokers.
Regards,Virendra
Re: Drop Topic from Consumer
Posted by Mayuresh Gharat <gh...@gmail.com>.
Np. :)
Thanks,
Mayuresh
On Wed, Oct 28, 2015 at 2:27 PM, Casey Daniell <ca...@hotmail.com>
wrote:
> Thanks Mayuresh.
> I was able to use Zookeeper's zkCli.sh and find the offsets / owners for
> the consumer that's no longer using a topicand delete them!
> Issue solved. thanks again.> Date: Tue, 27 Oct 2015 10:17:12 -0700
> > Subject: Re: Drop Topic from Consumer
> > From: gharatmayuresh15@gmail.com
> > To: users@kafka.apache.org
> >
> > Is it a wildcard consumer or you specify the topics manually?
> > You can do this by modifying the znode in zookeeper for that consumer
> group.
> >
> > Thanks,
> >
> > Mayuresh
> >
> > On Tue, Oct 27, 2015 at 8:09 AM, Casey Daniell <
> casey_daniell@hotmail.com>
> > wrote:
> >
> > > We are using Kafka 0.8.1 and have a Consumer that used to pull multiple
> > > topics. One of these topics is no longer needed by this
> consumer/group, so
> > > when I look at
> > > /usr/local/kafka/bin/kafka-run-class.sh
> kafka.tools.ConsumerOffsetChecker
> > > --zkconnect localhost --group <ServerName>
> > > We see the LAG for this measurement is high, but we know why. Is there
> > > anyway to let ZK know that this consumer group will never catch up for
> this
> > > topic, since it's not consumed any longer?
> > >
> >
> >
> >
> >
> > --
> > -Regards,
> > Mayuresh R. Gharat
> > (862) 250-7125
>
>
--
-Regards,
Mayuresh R. Gharat
(862) 250-7125
RE: Drop Topic from Consumer
Posted by Casey Daniell <ca...@hotmail.com>.
Thanks Mayuresh.
I was able to use Zookeeper's zkCli.sh and find the offsets / owners for the consumer that's no longer using a topicand delete them!
Issue solved. thanks again.> Date: Tue, 27 Oct 2015 10:17:12 -0700
> Subject: Re: Drop Topic from Consumer
> From: gharatmayuresh15@gmail.com
> To: users@kafka.apache.org
>
> Is it a wildcard consumer or you specify the topics manually?
> You can do this by modifying the znode in zookeeper for that consumer group.
>
> Thanks,
>
> Mayuresh
>
> On Tue, Oct 27, 2015 at 8:09 AM, Casey Daniell <ca...@hotmail.com>
> wrote:
>
> > We are using Kafka 0.8.1 and have a Consumer that used to pull multiple
> > topics. One of these topics is no longer needed by this consumer/group, so
> > when I look at
> > /usr/local/kafka/bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker
> > --zkconnect localhost --group <ServerName>
> > We see the LAG for this measurement is high, but we know why. Is there
> > anyway to let ZK know that this consumer group will never catch up for this
> > topic, since it's not consumed any longer?
> >
>
>
>
>
> --
> -Regards,
> Mayuresh R. Gharat
> (862) 250-7125
Re: Drop Topic from Consumer
Posted by Mayuresh Gharat <gh...@gmail.com>.
Is it a wildcard consumer or you specify the topics manually?
You can do this by modifying the znode in zookeeper for that consumer group.
Thanks,
Mayuresh
On Tue, Oct 27, 2015 at 8:09 AM, Casey Daniell <ca...@hotmail.com>
wrote:
> We are using Kafka 0.8.1 and have a Consumer that used to pull multiple
> topics. One of these topics is no longer needed by this consumer/group, so
> when I look at
> /usr/local/kafka/bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker
> --zkconnect localhost --group <ServerName>
> We see the LAG for this measurement is high, but we know why. Is there
> anyway to let ZK know that this consumer group will never catch up for this
> topic, since it's not consumed any longer?
>
--
-Regards,
Mayuresh R. Gharat
(862) 250-7125