You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by "Xiang Zhou (Samuel)" <zh...@gmail.com> on 2015/06/29 07:50:43 UTC
How to fetch offset in SimpleConsumer using Java
Hi,
I use the following snippets to try to get fetch the offset in a
SimpleConsumer I have committed (the commission has created topic
__consumer_offsets):
fetchRequest = new OffsetFetchRequest(
clientGroup, // What's the client group for simple consumer?
partitions,
(short) 1 /* version */, // version 1 and above fetch from Kafka, version
0 fetches from ZooKeeper
correlationId,
clientId);
channel.send(fetchRequest.underlying());
OffsetFetchResponse fetchResponse = OffsetFetchResponse.readFrom(channel
.receive().buffer());
But it keeps getting offset with value -1 and error code 3(no such
topic/partition found). Does anybody know if it is possible to use
OffsetFetchRequest to get offset for simple consumer because it requires
client group? If it can, could anybody suggest me how to fetch it?
Thanks!
Samuel
Re: How to fetch offset in SimpleConsumer using Java
Posted by "Xiang Zhou (Samuel)" <zh...@gmail.com>.
I found the cause of the problem at last, the version of kafka jar on
client side is 0.8.2.0. If you look at
OffsetFetchRequest.scala of 0.8.2.0. Every time the fetchRequest was
trying to fetch offset from zookeeper(the version number when fetching ONLY
support 0 on 0.8.2.0, but the commission on the same version can be set to
version number 1 !!). So updating kafka jar 0.8.2.1 solved the problem.
On Mon, Jun 29, 2015 at 11:10 AM, Xiang Zhou (Samuel) <zh...@gmail.com>
wrote:
> Thank you for your response Noah. I just tried with the same groupId and
> group name when I committed the offset, it still got the offset value -1
> and error code 3(UnknownTopicOrPartition). But I could get
> the TopicMetadataResponse for the topic and partition with this channel so
> the error code 3 makes me confused. I noticed there are some messages saved
> in __consumer_offsets when committing offsets. How could I check the
> contents in topic __consumer_offsets to see if the content is correct there?
>
> Thanks again!
>
> On Mon, Jun 29, 2015 at 3:35 AM, noah <ia...@gmail.com> wrote:
>
>> I believe clientGroup is your consumer group id. You must've picked a
>> value
>> to commit with, so it needs to be the same one.
>>
>> On Mon, Jun 29, 2015 at 12:50 AM Xiang Zhou (Samuel) <zh...@gmail.com>
>> wrote:
>>
>> > Hi,
>> >
>> > I use the following snippets to try to get fetch the offset in a
>> > SimpleConsumer I have committed (the commission has created topic
>> > __consumer_offsets):
>> >
>> > fetchRequest = new OffsetFetchRequest(
>> >
>> > clientGroup, // What's the client group for simple consumer?
>> >
>> > partitions,
>> >
>> > (short) 1 /* version */, // version 1 and above fetch from Kafka,
>> version
>> > 0 fetches from ZooKeeper
>> >
>> > correlationId,
>> >
>> > clientId);
>> >
>> > channel.send(fetchRequest.underlying());
>> >
>> > OffsetFetchResponse fetchResponse =
>> OffsetFetchResponse.readFrom(channel
>> > .receive().buffer());
>> >
>> > But it keeps getting offset with value -1 and error code 3(no such
>> > topic/partition found). Does anybody know if it is possible to use
>> > OffsetFetchRequest to get offset for simple consumer because it requires
>> > client group? If it can, could anybody suggest me how to fetch it?
>> >
>> > Thanks!
>> >
>> > Samuel
>> >
>>
>
>
Re: How to fetch offset in SimpleConsumer using Java
Posted by "Xiang Zhou (Samuel)" <zh...@gmail.com>.
Thank you for your response Noah. I just tried with the same groupId and
group name when I committed the offset, it still got the offset value -1
and error code 3(UnknownTopicOrPartition). But I could get
the TopicMetadataResponse for the topic and partition with this channel so
the error code 3 makes me confused. I noticed there are some messages saved
in __consumer_offsets when committing offsets. How could I check the
contents in topic __consumer_offsets to see if the content is correct there?
Thanks again!
On Mon, Jun 29, 2015 at 3:35 AM, noah <ia...@gmail.com> wrote:
> I believe clientGroup is your consumer group id. You must've picked a value
> to commit with, so it needs to be the same one.
>
> On Mon, Jun 29, 2015 at 12:50 AM Xiang Zhou (Samuel) <zh...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I use the following snippets to try to get fetch the offset in a
> > SimpleConsumer I have committed (the commission has created topic
> > __consumer_offsets):
> >
> > fetchRequest = new OffsetFetchRequest(
> >
> > clientGroup, // What's the client group for simple consumer?
> >
> > partitions,
> >
> > (short) 1 /* version */, // version 1 and above fetch from Kafka,
> version
> > 0 fetches from ZooKeeper
> >
> > correlationId,
> >
> > clientId);
> >
> > channel.send(fetchRequest.underlying());
> >
> > OffsetFetchResponse fetchResponse = OffsetFetchResponse.readFrom(channel
> > .receive().buffer());
> >
> > But it keeps getting offset with value -1 and error code 3(no such
> > topic/partition found). Does anybody know if it is possible to use
> > OffsetFetchRequest to get offset for simple consumer because it requires
> > client group? If it can, could anybody suggest me how to fetch it?
> >
> > Thanks!
> >
> > Samuel
> >
>
Re: How to fetch offset in SimpleConsumer using Java
Posted by noah <ia...@gmail.com>.
I believe clientGroup is your consumer group id. You must've picked a value
to commit with, so it needs to be the same one.
On Mon, Jun 29, 2015 at 12:50 AM Xiang Zhou (Samuel) <zh...@gmail.com>
wrote:
> Hi,
>
> I use the following snippets to try to get fetch the offset in a
> SimpleConsumer I have committed (the commission has created topic
> __consumer_offsets):
>
> fetchRequest = new OffsetFetchRequest(
>
> clientGroup, // What's the client group for simple consumer?
>
> partitions,
>
> (short) 1 /* version */, // version 1 and above fetch from Kafka, version
> 0 fetches from ZooKeeper
>
> correlationId,
>
> clientId);
>
> channel.send(fetchRequest.underlying());
>
> OffsetFetchResponse fetchResponse = OffsetFetchResponse.readFrom(channel
> .receive().buffer());
>
> But it keeps getting offset with value -1 and error code 3(no such
> topic/partition found). Does anybody know if it is possible to use
> OffsetFetchRequest to get offset for simple consumer because it requires
> client group? If it can, could anybody suggest me how to fetch it?
>
> Thanks!
>
> Samuel
>