You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by navneet sharma <na...@gmail.com> on 2012/06/05 15:33:48 UTC
Consumer exception
Hi,
I downloaded the latest code from trunk last week and i was trying this
snippet as given in examples.
Map<String, Integer> topicCountMap = new HashMap<String, Integer>();
topicCountMap.put(topic, new Integer(1));
Map<String, List<KafkaStream<Message>>> consumerMap =
consumer.createMessageStreams(topicCountMap);
KafkaStream<Message> stream = consumerMap.get(topic).get(0);
ConsumerIterator<Message> it = stream.iterator();
while(it.hasNext())
System.out.println(ExampleUtils.getMessage(it.next().message()));
}
But its giving following error:
Type mismatch: cannot convert from KafkaStream<MessageAndMetadata> to
KafkaStream<Message> at
ConsumerIterator<Message> it = stream.iterator();
Can you please tell how to run the above snippet with MessageAndMetadata?
And also please tell whether it requires any code changes in producer as
well?
Thanks,
Navneet Sharma
Re: Consumer exception
Posted by Jun Rao <ju...@gmail.com>.
MessageAndMetadata contains a Message and the metadata associated with it
(e.g., topic, partition, offset).
Thanks,
Jun
On Wed, Jun 6, 2012 at 10:59 AM, navneet sharma <navneetsharma0505@gmail.com
> wrote:
> Thanks Jun, i got it.
> Somehow i overlooked "iter.next().message()" .
>
> What confused me more is that when iterator is ConsumerIterator<Message> ,
> how its returning MessageAndMetadata<Message>.
> Whats the relation between Message and MessageAndMetadata?
>
> Regards,
> Navneet Sharma
>
>
> On Wed, Jun 6, 2012 at 8:28 PM, Jun Rao <ju...@gmail.com> wrote:
>
> > Navneet,
> >
> > KafkaStream<Message> returns an iterator of MessageAndMetadata<Message>.
> > The java example simply calls iter.next().message() to get the message
> from
> > MessageAndMetadata.
> >
> > Thanks,
> >
> > Jun
> >
> > On Tue, Jun 5, 2012 at 11:13 PM, navneet sharma <
> > navneetsharma0505@gmail.com
> > > wrote:
> >
> > > Hi Jun,
> > >
> > > Just to make sure that i have updated code, i updated source code just
> > now
> > > from this location:
> > > http://svn.apache.org/repos/asf/incubator/kafka/trunk
> > >
> > > but i am not seeing kafka.examples.Consumer with MessageAndMetadata
> > > implementation.
> > > Any help will be appreciated. Meanwhile, I will try something on my
> own.
> > >
> > > Thanks,
> > > Navneet Sharma
> > >
> > > On Wed, Jun 6, 2012 at 9:49 AM, Jun Rao <ju...@gmail.com> wrote:
> > >
> > > > Java example in trunk is modified to use MessageAndMetadata. This
> > > happened
> > > > probably 2-3 weeks ago.
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > > On Tue, Jun 5, 2012 at 8:49 PM, navneet sharma
> > > > <na...@gmail.com>wrote:
> > > >
> > > > > yes of course, the code snippet which i posted is from the
> > > > > kafka.examples.Consumer
> > > > > only.
> > > > >
> > > > > Its not modified to use MessageAndMetadata.
> > > > >
> > > > > Has it been changed in last week?
> > > > >
> > > > > Thanks,
> > > > > Navneet Sharma
> > > > >
> > > > > On Tue, Jun 5, 2012 at 8:12 PM, Jun Rao <ju...@gmail.com> wrote:
> > > > >
> > > > > > In trunk, the iterator api has changed to
> > > > > KafkaStream<MessageAndMetadata>.
> > > > > > Take a look at kafka.examples.Consumer in java-examples.
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > > On Tue, Jun 5, 2012 at 6:33 AM, navneet sharma
> > > > > > <na...@gmail.com>wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I downloaded the latest code from trunk last week and i was
> > trying
> > > > this
> > > > > > > snippet as given in examples.
> > > > > > >
> > > > > > >
> > > > > > > Map<String, Integer> topicCountMap = new HashMap<String,
> > > Integer>();
> > > > > > > topicCountMap.put(topic, new Integer(1));
> > > > > > > Map<String, List<KafkaStream<Message>>> consumerMap =
> > > > > > > consumer.createMessageStreams(topicCountMap);
> > > > > > > KafkaStream<Message> stream =
> consumerMap.get(topic).get(0);
> > > > > > > ConsumerIterator<Message> it = stream.iterator();
> > > > > > > while(it.hasNext())
> > > > > > >
> > > > System.out.println(ExampleUtils.getMessage(it.next().message()));
> > > > > > > }
> > > > > > >
> > > > > > > But its giving following error:
> > > > > > > Type mismatch: cannot convert from
> > KafkaStream<MessageAndMetadata>
> > > to
> > > > > > > KafkaStream<Message> at
> > > > > > > ConsumerIterator<Message> it = stream.iterator();
> > > > > > >
> > > > > > > Can you please tell how to run the above snippet with
> > > > > MessageAndMetadata?
> > > > > > > And also please tell whether it requires any code changes in
> > > producer
> > > > > as
> > > > > > > well?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Navneet Sharma
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Consumer exception
Posted by navneet sharma <na...@gmail.com>.
Thanks Jun, i got it.
Somehow i overlooked "iter.next().message()" .
What confused me more is that when iterator is ConsumerIterator<Message> ,
how its returning MessageAndMetadata<Message>.
Whats the relation between Message and MessageAndMetadata?
Regards,
Navneet Sharma
On Wed, Jun 6, 2012 at 8:28 PM, Jun Rao <ju...@gmail.com> wrote:
> Navneet,
>
> KafkaStream<Message> returns an iterator of MessageAndMetadata<Message>.
> The java example simply calls iter.next().message() to get the message from
> MessageAndMetadata.
>
> Thanks,
>
> Jun
>
> On Tue, Jun 5, 2012 at 11:13 PM, navneet sharma <
> navneetsharma0505@gmail.com
> > wrote:
>
> > Hi Jun,
> >
> > Just to make sure that i have updated code, i updated source code just
> now
> > from this location:
> > http://svn.apache.org/repos/asf/incubator/kafka/trunk
> >
> > but i am not seeing kafka.examples.Consumer with MessageAndMetadata
> > implementation.
> > Any help will be appreciated. Meanwhile, I will try something on my own.
> >
> > Thanks,
> > Navneet Sharma
> >
> > On Wed, Jun 6, 2012 at 9:49 AM, Jun Rao <ju...@gmail.com> wrote:
> >
> > > Java example in trunk is modified to use MessageAndMetadata. This
> > happened
> > > probably 2-3 weeks ago.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Tue, Jun 5, 2012 at 8:49 PM, navneet sharma
> > > <na...@gmail.com>wrote:
> > >
> > > > yes of course, the code snippet which i posted is from the
> > > > kafka.examples.Consumer
> > > > only.
> > > >
> > > > Its not modified to use MessageAndMetadata.
> > > >
> > > > Has it been changed in last week?
> > > >
> > > > Thanks,
> > > > Navneet Sharma
> > > >
> > > > On Tue, Jun 5, 2012 at 8:12 PM, Jun Rao <ju...@gmail.com> wrote:
> > > >
> > > > > In trunk, the iterator api has changed to
> > > > KafkaStream<MessageAndMetadata>.
> > > > > Take a look at kafka.examples.Consumer in java-examples.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > > > On Tue, Jun 5, 2012 at 6:33 AM, navneet sharma
> > > > > <na...@gmail.com>wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I downloaded the latest code from trunk last week and i was
> trying
> > > this
> > > > > > snippet as given in examples.
> > > > > >
> > > > > >
> > > > > > Map<String, Integer> topicCountMap = new HashMap<String,
> > Integer>();
> > > > > > topicCountMap.put(topic, new Integer(1));
> > > > > > Map<String, List<KafkaStream<Message>>> consumerMap =
> > > > > > consumer.createMessageStreams(topicCountMap);
> > > > > > KafkaStream<Message> stream = consumerMap.get(topic).get(0);
> > > > > > ConsumerIterator<Message> it = stream.iterator();
> > > > > > while(it.hasNext())
> > > > > >
> > > System.out.println(ExampleUtils.getMessage(it.next().message()));
> > > > > > }
> > > > > >
> > > > > > But its giving following error:
> > > > > > Type mismatch: cannot convert from
> KafkaStream<MessageAndMetadata>
> > to
> > > > > > KafkaStream<Message> at
> > > > > > ConsumerIterator<Message> it = stream.iterator();
> > > > > >
> > > > > > Can you please tell how to run the above snippet with
> > > > MessageAndMetadata?
> > > > > > And also please tell whether it requires any code changes in
> > producer
> > > > as
> > > > > > well?
> > > > > >
> > > > > > Thanks,
> > > > > > Navneet Sharma
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Consumer exception
Posted by Jun Rao <ju...@gmail.com>.
Navneet,
KafkaStream<Message> returns an iterator of MessageAndMetadata<Message>.
The java example simply calls iter.next().message() to get the message from
MessageAndMetadata.
Thanks,
Jun
On Tue, Jun 5, 2012 at 11:13 PM, navneet sharma <navneetsharma0505@gmail.com
> wrote:
> Hi Jun,
>
> Just to make sure that i have updated code, i updated source code just now
> from this location:
> http://svn.apache.org/repos/asf/incubator/kafka/trunk
>
> but i am not seeing kafka.examples.Consumer with MessageAndMetadata
> implementation.
> Any help will be appreciated. Meanwhile, I will try something on my own.
>
> Thanks,
> Navneet Sharma
>
> On Wed, Jun 6, 2012 at 9:49 AM, Jun Rao <ju...@gmail.com> wrote:
>
> > Java example in trunk is modified to use MessageAndMetadata. This
> happened
> > probably 2-3 weeks ago.
> >
> > Thanks,
> >
> > Jun
> >
> > On Tue, Jun 5, 2012 at 8:49 PM, navneet sharma
> > <na...@gmail.com>wrote:
> >
> > > yes of course, the code snippet which i posted is from the
> > > kafka.examples.Consumer
> > > only.
> > >
> > > Its not modified to use MessageAndMetadata.
> > >
> > > Has it been changed in last week?
> > >
> > > Thanks,
> > > Navneet Sharma
> > >
> > > On Tue, Jun 5, 2012 at 8:12 PM, Jun Rao <ju...@gmail.com> wrote:
> > >
> > > > In trunk, the iterator api has changed to
> > > KafkaStream<MessageAndMetadata>.
> > > > Take a look at kafka.examples.Consumer in java-examples.
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > > On Tue, Jun 5, 2012 at 6:33 AM, navneet sharma
> > > > <na...@gmail.com>wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I downloaded the latest code from trunk last week and i was trying
> > this
> > > > > snippet as given in examples.
> > > > >
> > > > >
> > > > > Map<String, Integer> topicCountMap = new HashMap<String,
> Integer>();
> > > > > topicCountMap.put(topic, new Integer(1));
> > > > > Map<String, List<KafkaStream<Message>>> consumerMap =
> > > > > consumer.createMessageStreams(topicCountMap);
> > > > > KafkaStream<Message> stream = consumerMap.get(topic).get(0);
> > > > > ConsumerIterator<Message> it = stream.iterator();
> > > > > while(it.hasNext())
> > > > >
> > System.out.println(ExampleUtils.getMessage(it.next().message()));
> > > > > }
> > > > >
> > > > > But its giving following error:
> > > > > Type mismatch: cannot convert from KafkaStream<MessageAndMetadata>
> to
> > > > > KafkaStream<Message> at
> > > > > ConsumerIterator<Message> it = stream.iterator();
> > > > >
> > > > > Can you please tell how to run the above snippet with
> > > MessageAndMetadata?
> > > > > And also please tell whether it requires any code changes in
> producer
> > > as
> > > > > well?
> > > > >
> > > > > Thanks,
> > > > > Navneet Sharma
> > > > >
> > > >
> > >
> >
>
Re: Consumer exception
Posted by navneet sharma <na...@gmail.com>.
Hi Jun,
Just to make sure that i have updated code, i updated source code just now
from this location:
http://svn.apache.org/repos/asf/incubator/kafka/trunk
but i am not seeing kafka.examples.Consumer with MessageAndMetadata
implementation.
Any help will be appreciated. Meanwhile, I will try something on my own.
Thanks,
Navneet Sharma
On Wed, Jun 6, 2012 at 9:49 AM, Jun Rao <ju...@gmail.com> wrote:
> Java example in trunk is modified to use MessageAndMetadata. This happened
> probably 2-3 weeks ago.
>
> Thanks,
>
> Jun
>
> On Tue, Jun 5, 2012 at 8:49 PM, navneet sharma
> <na...@gmail.com>wrote:
>
> > yes of course, the code snippet which i posted is from the
> > kafka.examples.Consumer
> > only.
> >
> > Its not modified to use MessageAndMetadata.
> >
> > Has it been changed in last week?
> >
> > Thanks,
> > Navneet Sharma
> >
> > On Tue, Jun 5, 2012 at 8:12 PM, Jun Rao <ju...@gmail.com> wrote:
> >
> > > In trunk, the iterator api has changed to
> > KafkaStream<MessageAndMetadata>.
> > > Take a look at kafka.examples.Consumer in java-examples.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Tue, Jun 5, 2012 at 6:33 AM, navneet sharma
> > > <na...@gmail.com>wrote:
> > >
> > > > Hi,
> > > >
> > > > I downloaded the latest code from trunk last week and i was trying
> this
> > > > snippet as given in examples.
> > > >
> > > >
> > > > Map<String, Integer> topicCountMap = new HashMap<String, Integer>();
> > > > topicCountMap.put(topic, new Integer(1));
> > > > Map<String, List<KafkaStream<Message>>> consumerMap =
> > > > consumer.createMessageStreams(topicCountMap);
> > > > KafkaStream<Message> stream = consumerMap.get(topic).get(0);
> > > > ConsumerIterator<Message> it = stream.iterator();
> > > > while(it.hasNext())
> > > >
> System.out.println(ExampleUtils.getMessage(it.next().message()));
> > > > }
> > > >
> > > > But its giving following error:
> > > > Type mismatch: cannot convert from KafkaStream<MessageAndMetadata> to
> > > > KafkaStream<Message> at
> > > > ConsumerIterator<Message> it = stream.iterator();
> > > >
> > > > Can you please tell how to run the above snippet with
> > MessageAndMetadata?
> > > > And also please tell whether it requires any code changes in producer
> > as
> > > > well?
> > > >
> > > > Thanks,
> > > > Navneet Sharma
> > > >
> > >
> >
>
Re: Consumer exception
Posted by Jun Rao <ju...@gmail.com>.
Java example in trunk is modified to use MessageAndMetadata. This happened
probably 2-3 weeks ago.
Thanks,
Jun
On Tue, Jun 5, 2012 at 8:49 PM, navneet sharma
<na...@gmail.com>wrote:
> yes of course, the code snippet which i posted is from the
> kafka.examples.Consumer
> only.
>
> Its not modified to use MessageAndMetadata.
>
> Has it been changed in last week?
>
> Thanks,
> Navneet Sharma
>
> On Tue, Jun 5, 2012 at 8:12 PM, Jun Rao <ju...@gmail.com> wrote:
>
> > In trunk, the iterator api has changed to
> KafkaStream<MessageAndMetadata>.
> > Take a look at kafka.examples.Consumer in java-examples.
> >
> > Thanks,
> >
> > Jun
> >
> > On Tue, Jun 5, 2012 at 6:33 AM, navneet sharma
> > <na...@gmail.com>wrote:
> >
> > > Hi,
> > >
> > > I downloaded the latest code from trunk last week and i was trying this
> > > snippet as given in examples.
> > >
> > >
> > > Map<String, Integer> topicCountMap = new HashMap<String, Integer>();
> > > topicCountMap.put(topic, new Integer(1));
> > > Map<String, List<KafkaStream<Message>>> consumerMap =
> > > consumer.createMessageStreams(topicCountMap);
> > > KafkaStream<Message> stream = consumerMap.get(topic).get(0);
> > > ConsumerIterator<Message> it = stream.iterator();
> > > while(it.hasNext())
> > > System.out.println(ExampleUtils.getMessage(it.next().message()));
> > > }
> > >
> > > But its giving following error:
> > > Type mismatch: cannot convert from KafkaStream<MessageAndMetadata> to
> > > KafkaStream<Message> at
> > > ConsumerIterator<Message> it = stream.iterator();
> > >
> > > Can you please tell how to run the above snippet with
> MessageAndMetadata?
> > > And also please tell whether it requires any code changes in producer
> as
> > > well?
> > >
> > > Thanks,
> > > Navneet Sharma
> > >
> >
>
Re: Consumer exception
Posted by navneet sharma <na...@gmail.com>.
yes of course, the code snippet which i posted is from the
kafka.examples.Consumer
only.
Its not modified to use MessageAndMetadata.
Has it been changed in last week?
Thanks,
Navneet Sharma
On Tue, Jun 5, 2012 at 8:12 PM, Jun Rao <ju...@gmail.com> wrote:
> In trunk, the iterator api has changed to KafkaStream<MessageAndMetadata>.
> Take a look at kafka.examples.Consumer in java-examples.
>
> Thanks,
>
> Jun
>
> On Tue, Jun 5, 2012 at 6:33 AM, navneet sharma
> <na...@gmail.com>wrote:
>
> > Hi,
> >
> > I downloaded the latest code from trunk last week and i was trying this
> > snippet as given in examples.
> >
> >
> > Map<String, Integer> topicCountMap = new HashMap<String, Integer>();
> > topicCountMap.put(topic, new Integer(1));
> > Map<String, List<KafkaStream<Message>>> consumerMap =
> > consumer.createMessageStreams(topicCountMap);
> > KafkaStream<Message> stream = consumerMap.get(topic).get(0);
> > ConsumerIterator<Message> it = stream.iterator();
> > while(it.hasNext())
> > System.out.println(ExampleUtils.getMessage(it.next().message()));
> > }
> >
> > But its giving following error:
> > Type mismatch: cannot convert from KafkaStream<MessageAndMetadata> to
> > KafkaStream<Message> at
> > ConsumerIterator<Message> it = stream.iterator();
> >
> > Can you please tell how to run the above snippet with MessageAndMetadata?
> > And also please tell whether it requires any code changes in producer as
> > well?
> >
> > Thanks,
> > Navneet Sharma
> >
>
Re: Consumer exception
Posted by Jun Rao <ju...@gmail.com>.
In trunk, the iterator api has changed to KafkaStream<MessageAndMetadata>.
Take a look at kafka.examples.Consumer in java-examples.
Thanks,
Jun
On Tue, Jun 5, 2012 at 6:33 AM, navneet sharma
<na...@gmail.com>wrote:
> Hi,
>
> I downloaded the latest code from trunk last week and i was trying this
> snippet as given in examples.
>
>
> Map<String, Integer> topicCountMap = new HashMap<String, Integer>();
> topicCountMap.put(topic, new Integer(1));
> Map<String, List<KafkaStream<Message>>> consumerMap =
> consumer.createMessageStreams(topicCountMap);
> KafkaStream<Message> stream = consumerMap.get(topic).get(0);
> ConsumerIterator<Message> it = stream.iterator();
> while(it.hasNext())
> System.out.println(ExampleUtils.getMessage(it.next().message()));
> }
>
> But its giving following error:
> Type mismatch: cannot convert from KafkaStream<MessageAndMetadata> to
> KafkaStream<Message> at
> ConsumerIterator<Message> it = stream.iterator();
>
> Can you please tell how to run the above snippet with MessageAndMetadata?
> And also please tell whether it requires any code changes in producer as
> well?
>
> Thanks,
> Navneet Sharma
>