You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Cliff Rhyne <cr...@signal.co> on 2016/05/08 20:26:32 UTC
reading the consumer offsets topic
I'm having difficulty reading the consumer offsets topic from the command
line. I try the following but it doesn't seem to work (along with a few
related variants including specifying the zookeeper hosts):
./kafka-console-consumer.sh --broker-list localhost:9092 --topic
__consumer_offsets
Is there a special way to read the consumer offsets topic?
Thanks,
Cliff
--
Cliff Rhyne
Software Engineering Manager
e: crhyne@signal.co
signal.co
________________________
Cut Through the Noise
This e-mail and any files transmitted with it are for the sole use of the
intended recipient(s) and may contain confidential and privileged
information. Any unauthorized use of this email is strictly prohibited.
©2016 Signal. All rights reserved.
Re: reading the consumer offsets topic
Posted by Cliff Rhyne <cr...@signal.co>.
Hi Tao,
Sorry for the delay. Thanks for pointing out that property. That was the
fix.
On Mon, May 9, 2016 at 6:00 PM, tao xiao <xi...@gmail.com> wrote:
> You need to enable internal topic in the consumer.properties
>
> exclude.internal.topics=false
>
> On Mon, 9 May 2016 at 12:42 Cliff Rhyne <cr...@signal.co> wrote:
>
> > Thanks Todd and Tao. I've tried those tricks but no luck.
> >
> > Just to add more info, this is the __consumer_offsets specific config
> that
> > is shown by the topic describe command:
> >
> >
> >
> Configs:segment.bytes=104857600,cleanup.policy=compact,compression.type=uncompressed
> >
> > On Mon, May 9, 2016 at 1:16 PM, tao xiao <xi...@gmail.com> wrote:
> >
> > > You can try this
> > >
> > > bin/kafka-console-consumer.sh --consumer.config
> > > config/consumer.properties --from-beginning
> > > --topic __consumer_offsets --zookeeper localhost:2181 —formatter
> > > "kafka.coordinator.GroupMetadataManager\$OffsetsMessageFormatter"
> > >
> > > On Mon, 9 May 2016 at 09:40 Todd Palino <tp...@gmail.com> wrote:
> > >
> > > > The GroupMetadataManager one should be working for you with 0.9. I
> > don’t
> > > > have a 0.9 KCC set up at the moment, so I’m using an 0.8 version
> where
> > > it’s
> > > > different (it’s the other class for me). The only thing I can offer
> now
> > > is
> > > > did you put quotes around the arg to --formatter so you don’t get
> weird
> > > > shell interference?
> > > >
> > > > -Todd
> > > >
> > > >
> > > > On Mon, May 9, 2016 at 8:18 AM, Cliff Rhyne <cr...@signal.co>
> wrote:
> > > >
> > > > > Thanks, Todd. It's still not working unfortunately.
> > > > >
> > > > > This results in nothing getting printed to the console and requires
> > > kill
> > > > -9
> > > > > in another window to stop (ctrl-c doesn't work):
> > > > >
> > > > > /kafka-console-consumer.sh --bootstrap-server localhost:9092
> > > --zookeeper
> > > > > <zkhost> --topic __consumer_offsets --formatter
> > > > > kafka.coordinator.GroupMetadataManager\$OffsetsMessageFormatter
> > > > >
> > > > > This results in a stack trace because it can't find the class:
> > > > >
> > > > > ./kafka-console-consumer.sh --bootstrap-server localhost:9092
> > > --zookeeper
> > > > > <zkhost> --topic __consumer_offsets --formatter
> > > > > kafka.server.OffsetManager\$OffsetsMessageFormatter
> > > > >
> > > > > Exception in thread "main" java.lang.ClassNotFoundException:
> > > > > kafka.server.OffsetManager$OffsetsMessageFormatter
> > > > >
> > > > >
> > > > > I'm on 0.9.0.1. "broker-list" is invalid and zookeeper is required
> > > > > regardless of the bootstrap-server parameter.
> > > > >
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Cliff
> > > > >
> > > > > On Sun, May 8, 2016 at 7:35 PM, Todd Palino <tp...@gmail.com>
> > wrote:
> > > > >
> > > > > > It looks like you’re just missing the proper message formatter.
> Of
> > > > > course,
> > > > > > that largely depends on your version of the broker. Try:
> > > > > >
> > > > > > ./kafka-console-consumer.sh --broker-list localhost:9092 --topic
> > > > > > __consumer_offsets
> > > > > > --formatter
> > > > > kafka.coordinator.GroupMetadataManager\$OffsetsMessageFormatter
> > > > > >
> > > > > >
> > > > > > If for some reason that doesn’t work, you can try
> > > > > > "kafka.server.OffsetManager\$OffsetsMessageFormatter” instead.
> > > > > >
> > > > > > -Todd
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Sun, May 8, 2016 at 1:26 PM, Cliff Rhyne <cr...@signal.co>
> > > wrote:
> > > > > >
> > > > > > > I'm having difficulty reading the consumer offsets topic from
> the
> > > > > command
> > > > > > > line. I try the following but it doesn't seem to work (along
> > with
> > > a
> > > > > few
> > > > > > > related variants including specifying the zookeeper hosts):
> > > > > > >
> > > > > > > ./kafka-console-consumer.sh --broker-list localhost:9092
> --topic
> > > > > > > __consumer_offsets
> > > > > > >
> > > > > > > Is there a special way to read the consumer offsets topic?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Cliff
> > > > > > >
> > > > > > > --
> > > > > > > Cliff Rhyne
> > > > > > > Software Engineering Manager
> > > > > > > e: crhyne@signal.co
> > > > > > > signal.co
> > > > > > > ________________________
> > > > > > >
> > > > > > > Cut Through the Noise
> > > > > > >
> > > > > > > This e-mail and any files transmitted with it are for the sole
> > use
> > > of
> > > > > the
> > > > > > > intended recipient(s) and may contain confidential and
> privileged
> > > > > > > information. Any unauthorized use of this email is strictly
> > > > prohibited.
> > > > > > > ©2016 Signal. All rights reserved.
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > *—-*
> > > > > > *Todd Palino*
> > > > > > Staff Site Reliability Engineer
> > > > > > Data Infrastructure Streaming
> > > > > >
> > > > > >
> > > > > >
> > > > > > linkedin.com/in/toddpalino
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Cliff Rhyne
> > > > > Software Engineering Manager
> > > > > e: crhyne@signal.co
> > > > > signal.co
> > > > > ________________________
> > > > >
> > > > > Cut Through the Noise
> > > > >
> > > > > This e-mail and any files transmitted with it are for the sole use
> of
> > > the
> > > > > intended recipient(s) and may contain confidential and privileged
> > > > > information. Any unauthorized use of this email is strictly
> > prohibited.
> > > > > ©2016 Signal. All rights reserved.
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > *—-*
> > > > *Todd Palino*
> > > > Staff Site Reliability Engineer
> > > > Data Infrastructure Streaming
> > > >
> > > >
> > > >
> > > > linkedin.com/in/toddpalino
> > > >
> > >
> >
> >
> >
> > --
> > Cliff Rhyne
> > Software Engineering Manager
> > e: crhyne@signal.co
> > signal.co
> > ________________________
> >
> > Cut Through the Noise
> >
> > This e-mail and any files transmitted with it are for the sole use of the
> > intended recipient(s) and may contain confidential and privileged
> > information. Any unauthorized use of this email is strictly prohibited.
> > ©2016 Signal. All rights reserved.
> >
>
--
Cliff Rhyne
Software Engineering Manager
e: crhyne@signal.co
signal.co
________________________
Cut Through the Noise
This e-mail and any files transmitted with it are for the sole use of the
intended recipient(s) and may contain confidential and privileged
information. Any unauthorized use of this email is strictly prohibited.
©2016 Signal. All rights reserved.
Re: reading the consumer offsets topic
Posted by tao xiao <xi...@gmail.com>.
You need to enable internal topic in the consumer.properties
exclude.internal.topics=false
On Mon, 9 May 2016 at 12:42 Cliff Rhyne <cr...@signal.co> wrote:
> Thanks Todd and Tao. I've tried those tricks but no luck.
>
> Just to add more info, this is the __consumer_offsets specific config that
> is shown by the topic describe command:
>
>
> Configs:segment.bytes=104857600,cleanup.policy=compact,compression.type=uncompressed
>
> On Mon, May 9, 2016 at 1:16 PM, tao xiao <xi...@gmail.com> wrote:
>
> > You can try this
> >
> > bin/kafka-console-consumer.sh --consumer.config
> > config/consumer.properties --from-beginning
> > --topic __consumer_offsets --zookeeper localhost:2181 —formatter
> > "kafka.coordinator.GroupMetadataManager\$OffsetsMessageFormatter"
> >
> > On Mon, 9 May 2016 at 09:40 Todd Palino <tp...@gmail.com> wrote:
> >
> > > The GroupMetadataManager one should be working for you with 0.9. I
> don’t
> > > have a 0.9 KCC set up at the moment, so I’m using an 0.8 version where
> > it’s
> > > different (it’s the other class for me). The only thing I can offer now
> > is
> > > did you put quotes around the arg to --formatter so you don’t get weird
> > > shell interference?
> > >
> > > -Todd
> > >
> > >
> > > On Mon, May 9, 2016 at 8:18 AM, Cliff Rhyne <cr...@signal.co> wrote:
> > >
> > > > Thanks, Todd. It's still not working unfortunately.
> > > >
> > > > This results in nothing getting printed to the console and requires
> > kill
> > > -9
> > > > in another window to stop (ctrl-c doesn't work):
> > > >
> > > > /kafka-console-consumer.sh --bootstrap-server localhost:9092
> > --zookeeper
> > > > <zkhost> --topic __consumer_offsets --formatter
> > > > kafka.coordinator.GroupMetadataManager\$OffsetsMessageFormatter
> > > >
> > > > This results in a stack trace because it can't find the class:
> > > >
> > > > ./kafka-console-consumer.sh --bootstrap-server localhost:9092
> > --zookeeper
> > > > <zkhost> --topic __consumer_offsets --formatter
> > > > kafka.server.OffsetManager\$OffsetsMessageFormatter
> > > >
> > > > Exception in thread "main" java.lang.ClassNotFoundException:
> > > > kafka.server.OffsetManager$OffsetsMessageFormatter
> > > >
> > > >
> > > > I'm on 0.9.0.1. "broker-list" is invalid and zookeeper is required
> > > > regardless of the bootstrap-server parameter.
> > > >
> > > >
> > > > Thanks,
> > > >
> > > > Cliff
> > > >
> > > > On Sun, May 8, 2016 at 7:35 PM, Todd Palino <tp...@gmail.com>
> wrote:
> > > >
> > > > > It looks like you’re just missing the proper message formatter. Of
> > > > course,
> > > > > that largely depends on your version of the broker. Try:
> > > > >
> > > > > ./kafka-console-consumer.sh --broker-list localhost:9092 --topic
> > > > > __consumer_offsets
> > > > > --formatter
> > > > kafka.coordinator.GroupMetadataManager\$OffsetsMessageFormatter
> > > > >
> > > > >
> > > > > If for some reason that doesn’t work, you can try
> > > > > "kafka.server.OffsetManager\$OffsetsMessageFormatter” instead.
> > > > >
> > > > > -Todd
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Sun, May 8, 2016 at 1:26 PM, Cliff Rhyne <cr...@signal.co>
> > wrote:
> > > > >
> > > > > > I'm having difficulty reading the consumer offsets topic from the
> > > > command
> > > > > > line. I try the following but it doesn't seem to work (along
> with
> > a
> > > > few
> > > > > > related variants including specifying the zookeeper hosts):
> > > > > >
> > > > > > ./kafka-console-consumer.sh --broker-list localhost:9092 --topic
> > > > > > __consumer_offsets
> > > > > >
> > > > > > Is there a special way to read the consumer offsets topic?
> > > > > >
> > > > > > Thanks,
> > > > > > Cliff
> > > > > >
> > > > > > --
> > > > > > Cliff Rhyne
> > > > > > Software Engineering Manager
> > > > > > e: crhyne@signal.co
> > > > > > signal.co
> > > > > > ________________________
> > > > > >
> > > > > > Cut Through the Noise
> > > > > >
> > > > > > This e-mail and any files transmitted with it are for the sole
> use
> > of
> > > > the
> > > > > > intended recipient(s) and may contain confidential and privileged
> > > > > > information. Any unauthorized use of this email is strictly
> > > prohibited.
> > > > > > ©2016 Signal. All rights reserved.
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > *—-*
> > > > > *Todd Palino*
> > > > > Staff Site Reliability Engineer
> > > > > Data Infrastructure Streaming
> > > > >
> > > > >
> > > > >
> > > > > linkedin.com/in/toddpalino
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Cliff Rhyne
> > > > Software Engineering Manager
> > > > e: crhyne@signal.co
> > > > signal.co
> > > > ________________________
> > > >
> > > > Cut Through the Noise
> > > >
> > > > This e-mail and any files transmitted with it are for the sole use of
> > the
> > > > intended recipient(s) and may contain confidential and privileged
> > > > information. Any unauthorized use of this email is strictly
> prohibited.
> > > > ©2016 Signal. All rights reserved.
> > > >
> > >
> > >
> > >
> > > --
> > > *—-*
> > > *Todd Palino*
> > > Staff Site Reliability Engineer
> > > Data Infrastructure Streaming
> > >
> > >
> > >
> > > linkedin.com/in/toddpalino
> > >
> >
>
>
>
> --
> Cliff Rhyne
> Software Engineering Manager
> e: crhyne@signal.co
> signal.co
> ________________________
>
> Cut Through the Noise
>
> This e-mail and any files transmitted with it are for the sole use of the
> intended recipient(s) and may contain confidential and privileged
> information. Any unauthorized use of this email is strictly prohibited.
> ©2016 Signal. All rights reserved.
>
Re: reading the consumer offsets topic
Posted by Cliff Rhyne <cr...@signal.co>.
Thanks Todd and Tao. I've tried those tricks but no luck.
Just to add more info, this is the __consumer_offsets specific config that
is shown by the topic describe command:
Configs:segment.bytes=104857600,cleanup.policy=compact,compression.type=uncompressed
On Mon, May 9, 2016 at 1:16 PM, tao xiao <xi...@gmail.com> wrote:
> You can try this
>
> bin/kafka-console-consumer.sh --consumer.config
> config/consumer.properties --from-beginning
> --topic __consumer_offsets --zookeeper localhost:2181 —formatter
> "kafka.coordinator.GroupMetadataManager\$OffsetsMessageFormatter"
>
> On Mon, 9 May 2016 at 09:40 Todd Palino <tp...@gmail.com> wrote:
>
> > The GroupMetadataManager one should be working for you with 0.9. I don’t
> > have a 0.9 KCC set up at the moment, so I’m using an 0.8 version where
> it’s
> > different (it’s the other class for me). The only thing I can offer now
> is
> > did you put quotes around the arg to --formatter so you don’t get weird
> > shell interference?
> >
> > -Todd
> >
> >
> > On Mon, May 9, 2016 at 8:18 AM, Cliff Rhyne <cr...@signal.co> wrote:
> >
> > > Thanks, Todd. It's still not working unfortunately.
> > >
> > > This results in nothing getting printed to the console and requires
> kill
> > -9
> > > in another window to stop (ctrl-c doesn't work):
> > >
> > > /kafka-console-consumer.sh --bootstrap-server localhost:9092
> --zookeeper
> > > <zkhost> --topic __consumer_offsets --formatter
> > > kafka.coordinator.GroupMetadataManager\$OffsetsMessageFormatter
> > >
> > > This results in a stack trace because it can't find the class:
> > >
> > > ./kafka-console-consumer.sh --bootstrap-server localhost:9092
> --zookeeper
> > > <zkhost> --topic __consumer_offsets --formatter
> > > kafka.server.OffsetManager\$OffsetsMessageFormatter
> > >
> > > Exception in thread "main" java.lang.ClassNotFoundException:
> > > kafka.server.OffsetManager$OffsetsMessageFormatter
> > >
> > >
> > > I'm on 0.9.0.1. "broker-list" is invalid and zookeeper is required
> > > regardless of the bootstrap-server parameter.
> > >
> > >
> > > Thanks,
> > >
> > > Cliff
> > >
> > > On Sun, May 8, 2016 at 7:35 PM, Todd Palino <tp...@gmail.com> wrote:
> > >
> > > > It looks like you’re just missing the proper message formatter. Of
> > > course,
> > > > that largely depends on your version of the broker. Try:
> > > >
> > > > ./kafka-console-consumer.sh --broker-list localhost:9092 --topic
> > > > __consumer_offsets
> > > > --formatter
> > > kafka.coordinator.GroupMetadataManager\$OffsetsMessageFormatter
> > > >
> > > >
> > > > If for some reason that doesn’t work, you can try
> > > > "kafka.server.OffsetManager\$OffsetsMessageFormatter” instead.
> > > >
> > > > -Todd
> > > >
> > > >
> > > >
> > > >
> > > > On Sun, May 8, 2016 at 1:26 PM, Cliff Rhyne <cr...@signal.co>
> wrote:
> > > >
> > > > > I'm having difficulty reading the consumer offsets topic from the
> > > command
> > > > > line. I try the following but it doesn't seem to work (along with
> a
> > > few
> > > > > related variants including specifying the zookeeper hosts):
> > > > >
> > > > > ./kafka-console-consumer.sh --broker-list localhost:9092 --topic
> > > > > __consumer_offsets
> > > > >
> > > > > Is there a special way to read the consumer offsets topic?
> > > > >
> > > > > Thanks,
> > > > > Cliff
> > > > >
> > > > > --
> > > > > Cliff Rhyne
> > > > > Software Engineering Manager
> > > > > e: crhyne@signal.co
> > > > > signal.co
> > > > > ________________________
> > > > >
> > > > > Cut Through the Noise
> > > > >
> > > > > This e-mail and any files transmitted with it are for the sole use
> of
> > > the
> > > > > intended recipient(s) and may contain confidential and privileged
> > > > > information. Any unauthorized use of this email is strictly
> > prohibited.
> > > > > ©2016 Signal. All rights reserved.
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > *—-*
> > > > *Todd Palino*
> > > > Staff Site Reliability Engineer
> > > > Data Infrastructure Streaming
> > > >
> > > >
> > > >
> > > > linkedin.com/in/toddpalino
> > > >
> > >
> > >
> > >
> > > --
> > > Cliff Rhyne
> > > Software Engineering Manager
> > > e: crhyne@signal.co
> > > signal.co
> > > ________________________
> > >
> > > Cut Through the Noise
> > >
> > > This e-mail and any files transmitted with it are for the sole use of
> the
> > > intended recipient(s) and may contain confidential and privileged
> > > information. Any unauthorized use of this email is strictly prohibited.
> > > ©2016 Signal. All rights reserved.
> > >
> >
> >
> >
> > --
> > *—-*
> > *Todd Palino*
> > Staff Site Reliability Engineer
> > Data Infrastructure Streaming
> >
> >
> >
> > linkedin.com/in/toddpalino
> >
>
--
Cliff Rhyne
Software Engineering Manager
e: crhyne@signal.co
signal.co
________________________
Cut Through the Noise
This e-mail and any files transmitted with it are for the sole use of the
intended recipient(s) and may contain confidential and privileged
information. Any unauthorized use of this email is strictly prohibited.
©2016 Signal. All rights reserved.
Re: reading the consumer offsets topic
Posted by tao xiao <xi...@gmail.com>.
You can try this
bin/kafka-console-consumer.sh --consumer.config
config/consumer.properties --from-beginning
--topic __consumer_offsets --zookeeper localhost:2181 —formatter
"kafka.coordinator.GroupMetadataManager\$OffsetsMessageFormatter"
On Mon, 9 May 2016 at 09:40 Todd Palino <tp...@gmail.com> wrote:
> The GroupMetadataManager one should be working for you with 0.9. I don’t
> have a 0.9 KCC set up at the moment, so I’m using an 0.8 version where it’s
> different (it’s the other class for me). The only thing I can offer now is
> did you put quotes around the arg to --formatter so you don’t get weird
> shell interference?
>
> -Todd
>
>
> On Mon, May 9, 2016 at 8:18 AM, Cliff Rhyne <cr...@signal.co> wrote:
>
> > Thanks, Todd. It's still not working unfortunately.
> >
> > This results in nothing getting printed to the console and requires kill
> -9
> > in another window to stop (ctrl-c doesn't work):
> >
> > /kafka-console-consumer.sh --bootstrap-server localhost:9092 --zookeeper
> > <zkhost> --topic __consumer_offsets --formatter
> > kafka.coordinator.GroupMetadataManager\$OffsetsMessageFormatter
> >
> > This results in a stack trace because it can't find the class:
> >
> > ./kafka-console-consumer.sh --bootstrap-server localhost:9092 --zookeeper
> > <zkhost> --topic __consumer_offsets --formatter
> > kafka.server.OffsetManager\$OffsetsMessageFormatter
> >
> > Exception in thread "main" java.lang.ClassNotFoundException:
> > kafka.server.OffsetManager$OffsetsMessageFormatter
> >
> >
> > I'm on 0.9.0.1. "broker-list" is invalid and zookeeper is required
> > regardless of the bootstrap-server parameter.
> >
> >
> > Thanks,
> >
> > Cliff
> >
> > On Sun, May 8, 2016 at 7:35 PM, Todd Palino <tp...@gmail.com> wrote:
> >
> > > It looks like you’re just missing the proper message formatter. Of
> > course,
> > > that largely depends on your version of the broker. Try:
> > >
> > > ./kafka-console-consumer.sh --broker-list localhost:9092 --topic
> > > __consumer_offsets
> > > --formatter
> > kafka.coordinator.GroupMetadataManager\$OffsetsMessageFormatter
> > >
> > >
> > > If for some reason that doesn’t work, you can try
> > > "kafka.server.OffsetManager\$OffsetsMessageFormatter” instead.
> > >
> > > -Todd
> > >
> > >
> > >
> > >
> > > On Sun, May 8, 2016 at 1:26 PM, Cliff Rhyne <cr...@signal.co> wrote:
> > >
> > > > I'm having difficulty reading the consumer offsets topic from the
> > command
> > > > line. I try the following but it doesn't seem to work (along with a
> > few
> > > > related variants including specifying the zookeeper hosts):
> > > >
> > > > ./kafka-console-consumer.sh --broker-list localhost:9092 --topic
> > > > __consumer_offsets
> > > >
> > > > Is there a special way to read the consumer offsets topic?
> > > >
> > > > Thanks,
> > > > Cliff
> > > >
> > > > --
> > > > Cliff Rhyne
> > > > Software Engineering Manager
> > > > e: crhyne@signal.co
> > > > signal.co
> > > > ________________________
> > > >
> > > > Cut Through the Noise
> > > >
> > > > This e-mail and any files transmitted with it are for the sole use of
> > the
> > > > intended recipient(s) and may contain confidential and privileged
> > > > information. Any unauthorized use of this email is strictly
> prohibited.
> > > > ©2016 Signal. All rights reserved.
> > > >
> > >
> > >
> > >
> > > --
> > > *—-*
> > > *Todd Palino*
> > > Staff Site Reliability Engineer
> > > Data Infrastructure Streaming
> > >
> > >
> > >
> > > linkedin.com/in/toddpalino
> > >
> >
> >
> >
> > --
> > Cliff Rhyne
> > Software Engineering Manager
> > e: crhyne@signal.co
> > signal.co
> > ________________________
> >
> > Cut Through the Noise
> >
> > This e-mail and any files transmitted with it are for the sole use of the
> > intended recipient(s) and may contain confidential and privileged
> > information. Any unauthorized use of this email is strictly prohibited.
> > ©2016 Signal. All rights reserved.
> >
>
>
>
> --
> *—-*
> *Todd Palino*
> Staff Site Reliability Engineer
> Data Infrastructure Streaming
>
>
>
> linkedin.com/in/toddpalino
>
Re: reading the consumer offsets topic
Posted by Todd Palino <tp...@gmail.com>.
The GroupMetadataManager one should be working for you with 0.9. I don’t
have a 0.9 KCC set up at the moment, so I’m using an 0.8 version where it’s
different (it’s the other class for me). The only thing I can offer now is
did you put quotes around the arg to --formatter so you don’t get weird
shell interference?
-Todd
On Mon, May 9, 2016 at 8:18 AM, Cliff Rhyne <cr...@signal.co> wrote:
> Thanks, Todd. It's still not working unfortunately.
>
> This results in nothing getting printed to the console and requires kill -9
> in another window to stop (ctrl-c doesn't work):
>
> /kafka-console-consumer.sh --bootstrap-server localhost:9092 --zookeeper
> <zkhost> --topic __consumer_offsets --formatter
> kafka.coordinator.GroupMetadataManager\$OffsetsMessageFormatter
>
> This results in a stack trace because it can't find the class:
>
> ./kafka-console-consumer.sh --bootstrap-server localhost:9092 --zookeeper
> <zkhost> --topic __consumer_offsets --formatter
> kafka.server.OffsetManager\$OffsetsMessageFormatter
>
> Exception in thread "main" java.lang.ClassNotFoundException:
> kafka.server.OffsetManager$OffsetsMessageFormatter
>
>
> I'm on 0.9.0.1. "broker-list" is invalid and zookeeper is required
> regardless of the bootstrap-server parameter.
>
>
> Thanks,
>
> Cliff
>
> On Sun, May 8, 2016 at 7:35 PM, Todd Palino <tp...@gmail.com> wrote:
>
> > It looks like you’re just missing the proper message formatter. Of
> course,
> > that largely depends on your version of the broker. Try:
> >
> > ./kafka-console-consumer.sh --broker-list localhost:9092 --topic
> > __consumer_offsets
> > --formatter
> kafka.coordinator.GroupMetadataManager\$OffsetsMessageFormatter
> >
> >
> > If for some reason that doesn’t work, you can try
> > "kafka.server.OffsetManager\$OffsetsMessageFormatter” instead.
> >
> > -Todd
> >
> >
> >
> >
> > On Sun, May 8, 2016 at 1:26 PM, Cliff Rhyne <cr...@signal.co> wrote:
> >
> > > I'm having difficulty reading the consumer offsets topic from the
> command
> > > line. I try the following but it doesn't seem to work (along with a
> few
> > > related variants including specifying the zookeeper hosts):
> > >
> > > ./kafka-console-consumer.sh --broker-list localhost:9092 --topic
> > > __consumer_offsets
> > >
> > > Is there a special way to read the consumer offsets topic?
> > >
> > > Thanks,
> > > Cliff
> > >
> > > --
> > > Cliff Rhyne
> > > Software Engineering Manager
> > > e: crhyne@signal.co
> > > signal.co
> > > ________________________
> > >
> > > Cut Through the Noise
> > >
> > > This e-mail and any files transmitted with it are for the sole use of
> the
> > > intended recipient(s) and may contain confidential and privileged
> > > information. Any unauthorized use of this email is strictly prohibited.
> > > ©2016 Signal. All rights reserved.
> > >
> >
> >
> >
> > --
> > *—-*
> > *Todd Palino*
> > Staff Site Reliability Engineer
> > Data Infrastructure Streaming
> >
> >
> >
> > linkedin.com/in/toddpalino
> >
>
>
>
> --
> Cliff Rhyne
> Software Engineering Manager
> e: crhyne@signal.co
> signal.co
> ________________________
>
> Cut Through the Noise
>
> This e-mail and any files transmitted with it are for the sole use of the
> intended recipient(s) and may contain confidential and privileged
> information. Any unauthorized use of this email is strictly prohibited.
> ©2016 Signal. All rights reserved.
>
--
*—-*
*Todd Palino*
Staff Site Reliability Engineer
Data Infrastructure Streaming
linkedin.com/in/toddpalino
Re: reading the consumer offsets topic
Posted by Cliff Rhyne <cr...@signal.co>.
Thanks, Todd. It's still not working unfortunately.
This results in nothing getting printed to the console and requires kill -9
in another window to stop (ctrl-c doesn't work):
/kafka-console-consumer.sh --bootstrap-server localhost:9092 --zookeeper
<zkhost> --topic __consumer_offsets --formatter
kafka.coordinator.GroupMetadataManager\$OffsetsMessageFormatter
This results in a stack trace because it can't find the class:
./kafka-console-consumer.sh --bootstrap-server localhost:9092 --zookeeper
<zkhost> --topic __consumer_offsets --formatter
kafka.server.OffsetManager\$OffsetsMessageFormatter
Exception in thread "main" java.lang.ClassNotFoundException:
kafka.server.OffsetManager$OffsetsMessageFormatter
I'm on 0.9.0.1. "broker-list" is invalid and zookeeper is required
regardless of the bootstrap-server parameter.
Thanks,
Cliff
On Sun, May 8, 2016 at 7:35 PM, Todd Palino <tp...@gmail.com> wrote:
> It looks like you’re just missing the proper message formatter. Of course,
> that largely depends on your version of the broker. Try:
>
> ./kafka-console-consumer.sh --broker-list localhost:9092 --topic
> __consumer_offsets
> --formatter kafka.coordinator.GroupMetadataManager\$OffsetsMessageFormatter
>
>
> If for some reason that doesn’t work, you can try
> "kafka.server.OffsetManager\$OffsetsMessageFormatter” instead.
>
> -Todd
>
>
>
>
> On Sun, May 8, 2016 at 1:26 PM, Cliff Rhyne <cr...@signal.co> wrote:
>
> > I'm having difficulty reading the consumer offsets topic from the command
> > line. I try the following but it doesn't seem to work (along with a few
> > related variants including specifying the zookeeper hosts):
> >
> > ./kafka-console-consumer.sh --broker-list localhost:9092 --topic
> > __consumer_offsets
> >
> > Is there a special way to read the consumer offsets topic?
> >
> > Thanks,
> > Cliff
> >
> > --
> > Cliff Rhyne
> > Software Engineering Manager
> > e: crhyne@signal.co
> > signal.co
> > ________________________
> >
> > Cut Through the Noise
> >
> > This e-mail and any files transmitted with it are for the sole use of the
> > intended recipient(s) and may contain confidential and privileged
> > information. Any unauthorized use of this email is strictly prohibited.
> > ©2016 Signal. All rights reserved.
> >
>
>
>
> --
> *—-*
> *Todd Palino*
> Staff Site Reliability Engineer
> Data Infrastructure Streaming
>
>
>
> linkedin.com/in/toddpalino
>
--
Cliff Rhyne
Software Engineering Manager
e: crhyne@signal.co
signal.co
________________________
Cut Through the Noise
This e-mail and any files transmitted with it are for the sole use of the
intended recipient(s) and may contain confidential and privileged
information. Any unauthorized use of this email is strictly prohibited.
©2016 Signal. All rights reserved.
Re: reading the consumer offsets topic
Posted by Todd Palino <tp...@gmail.com>.
It looks like you’re just missing the proper message formatter. Of course,
that largely depends on your version of the broker. Try:
./kafka-console-consumer.sh --broker-list localhost:9092 --topic
__consumer_offsets
--formatter kafka.coordinator.GroupMetadataManager\$OffsetsMessageFormatter
If for some reason that doesn’t work, you can try
"kafka.server.OffsetManager\$OffsetsMessageFormatter” instead.
-Todd
On Sun, May 8, 2016 at 1:26 PM, Cliff Rhyne <cr...@signal.co> wrote:
> I'm having difficulty reading the consumer offsets topic from the command
> line. I try the following but it doesn't seem to work (along with a few
> related variants including specifying the zookeeper hosts):
>
> ./kafka-console-consumer.sh --broker-list localhost:9092 --topic
> __consumer_offsets
>
> Is there a special way to read the consumer offsets topic?
>
> Thanks,
> Cliff
>
> --
> Cliff Rhyne
> Software Engineering Manager
> e: crhyne@signal.co
> signal.co
> ________________________
>
> Cut Through the Noise
>
> This e-mail and any files transmitted with it are for the sole use of the
> intended recipient(s) and may contain confidential and privileged
> information. Any unauthorized use of this email is strictly prohibited.
> ©2016 Signal. All rights reserved.
>
--
*—-*
*Todd Palino*
Staff Site Reliability Engineer
Data Infrastructure Streaming
linkedin.com/in/toddpalino