You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Victor L <vl...@gmail.com> on 2015/03/25 17:17:27 UTC
lost messages -?
Can someone let me know how to dump contents of topics?
I have producers sending messages to 3 brokers but about half of them don't
seem to be consumed. I suppose they are getting stuck in queues but how can
i figure out where?
Thks,
Re: lost messages -?
Posted by Mayuresh Gharat <gh...@gmail.com>.
You can use the DumpLogSegment tool.
Thanks,
Mayuresh
On Wed, Mar 25, 2015 at 9:17 AM, Victor L <vl...@gmail.com> wrote:
> Can someone let me know how to dump contents of topics?
> I have producers sending messages to 3 brokers but about half of them don't
> seem to be consumed. I suppose they are getting stuck in queues but how can
> i figure out where?
> Thks,
>
--
-Regards,
Mayuresh R. Gharat
(862) 250-7125
Re: lost messages -?
Posted by Harsha <ha...@fastmail.fm>.
Victor,
Its under kaka.tools.DumpLogSegments you can use kafka-run-class to execute it.
--
Harsha
On March 26, 2015 at 5:29:32 AM, Victor L (vlyamtsev@gmail.com) wrote:
Where's this tool (DumpLogSegments) in Kafka distro? Is it Java class in
kafka jar, or is it third party binary?
Thank you,
On Wed, Mar 25, 2015 at 1:11 PM, Mayuresh Gharat <gharatmayuresh15@gmail.com
> wrote:
> DumpLogSegments will give you output something like this :
>
> offset: 780613873770 isvalid: true payloadsize: 8055 magic: 1
> compresscodec:
> GZIPCompressionCodec
>
> If this is what you want you can use the tool, to detect if the messages
> are getting to your brokers.
> Console-Consumer will output the messages for you.
>
> Thanks,
>
> Mayuresh
>
>
> On Wed, Mar 25, 2015 at 9:33 AM, tao xiao <xi...@gmail.com> wrote:
>
> > You can use kafka-console-consumer consuming the topic from the beginning
> >
> > *kafka-console-consumer.sh --zookeeper localhost:2181 --topic test
> > --from-beginning*
> >
> >
> > On Thu, Mar 26, 2015 at 12:17 AM, Victor L <vl...@gmail.com> wrote:
> >
> > > Can someone let me know how to dump contents of topics?
> > > I have producers sending messages to 3 brokers but about half of them
> > don't
> > > seem to be consumed. I suppose they are getting stuck in queues but how
> > can
> > > i figure out where?
> > > Thks,
> > >
> >
> >
> >
> > --
> > Regards,
> > Tao
> >
>
>
>
> --
> -Regards,
> Mayuresh R. Gharat
> (862) 250-7125
>
Re: lost messages -?
Posted by Victor L <vl...@gmail.com>.
Where's this tool (DumpLogSegments) in Kafka distro? Is it Java class in
kafka jar, or is it third party binary?
Thank you,
On Wed, Mar 25, 2015 at 1:11 PM, Mayuresh Gharat <gharatmayuresh15@gmail.com
> wrote:
> DumpLogSegments will give you output something like this :
>
> offset: 780613873770 isvalid: true payloadsize: 8055 magic: 1
> compresscodec:
> GZIPCompressionCodec
>
> If this is what you want you can use the tool, to detect if the messages
> are getting to your brokers.
> Console-Consumer will output the messages for you.
>
> Thanks,
>
> Mayuresh
>
>
> On Wed, Mar 25, 2015 at 9:33 AM, tao xiao <xi...@gmail.com> wrote:
>
> > You can use kafka-console-consumer consuming the topic from the beginning
> >
> > *kafka-console-consumer.sh --zookeeper localhost:2181 --topic test
> > --from-beginning*
> >
> >
> > On Thu, Mar 26, 2015 at 12:17 AM, Victor L <vl...@gmail.com> wrote:
> >
> > > Can someone let me know how to dump contents of topics?
> > > I have producers sending messages to 3 brokers but about half of them
> > don't
> > > seem to be consumed. I suppose they are getting stuck in queues but how
> > can
> > > i figure out where?
> > > Thks,
> > >
> >
> >
> >
> > --
> > Regards,
> > Tao
> >
>
>
>
> --
> -Regards,
> Mayuresh R. Gharat
> (862) 250-7125
>
Re: lost messages -?
Posted by Mayuresh Gharat <gh...@gmail.com>.
DumpLogSegments will give you output something like this :
offset: 780613873770 isvalid: true payloadsize: 8055 magic: 1 compresscodec:
GZIPCompressionCodec
If this is what you want you can use the tool, to detect if the messages
are getting to your brokers.
Console-Consumer will output the messages for you.
Thanks,
Mayuresh
On Wed, Mar 25, 2015 at 9:33 AM, tao xiao <xi...@gmail.com> wrote:
> You can use kafka-console-consumer consuming the topic from the beginning
>
> *kafka-console-consumer.sh --zookeeper localhost:2181 --topic test
> --from-beginning*
>
>
> On Thu, Mar 26, 2015 at 12:17 AM, Victor L <vl...@gmail.com> wrote:
>
> > Can someone let me know how to dump contents of topics?
> > I have producers sending messages to 3 brokers but about half of them
> don't
> > seem to be consumed. I suppose they are getting stuck in queues but how
> can
> > i figure out where?
> > Thks,
> >
>
>
>
> --
> Regards,
> Tao
>
--
-Regards,
Mayuresh R. Gharat
(862) 250-7125
Re: lost messages -?
Posted by tao xiao <xi...@gmail.com>.
You can use kafka-console-consumer consuming the topic from the beginning
*kafka-console-consumer.sh --zookeeper localhost:2181 --topic test
--from-beginning*
On Thu, Mar 26, 2015 at 12:17 AM, Victor L <vl...@gmail.com> wrote:
> Can someone let me know how to dump contents of topics?
> I have producers sending messages to 3 brokers but about half of them don't
> seem to be consumed. I suppose they are getting stuck in queues but how can
> i figure out where?
> Thks,
>
--
Regards,
Tao