Kafka 0.8 High level consumer : Stream Iterator is blocking

[Ankit Jain <an...@impetus.co.in>]

Hi all,

I have a Kafka 0.8 cluster of two nodes on same machine with 4 partitions and communicating through a single zookeeper.

I am producing data using the Kafka Producer using the following code:
<code>
KeyedMessage<String, byte[]> data = new KeyedMessage<String, byte[]>(topic, input);
producer.send(data);
</code>

I am able to consume the data from a topic using the Kafka console consumer as well as simple consumer but it is blocking the iterator at High Level consumer.

References:
High Level Consumer - https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example
Simple Consumer - https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example

The configuration for High Level consumer is default configuration.

Thanks for your help in advance.

Regards,
Ankit Jain



________________________________






NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.

Re: Kafka 0.8 High level consumer : Stream Iterator is blocking

[Jun Rao <ju...@gmail.com>]

The consumer iterator by default blocks if there are no new messages. You
can configure it to be non-blocking. See consumer.timeout.ms
http://kafka.apache.org/08/configuration.html

Thanks,

Jun


On Wed, Jul 10, 2013 at 11:55 PM, Ankit Jain <an...@impetus.co.in>wrote:

> Hi all,
>
> I have a Kafka 0.8 cluster of two nodes on same machine with 4 partitions
> and communicating through a single zookeeper.
>
> I am producing data using the Kafka Producer using the following code:
> <code>
> KeyedMessage<String, byte[]> data = new KeyedMessage<String,
> byte[]>(topic, input);
> producer.send(data);
> </code>
>
> I am able to consume the data from a topic using the Kafka console
> consumer as well as simple consumer but it is blocking the iterator at High
> Level consumer.
>
> References:
> High Level Consumer -
> https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example
> Simple Consumer -
> https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example
>
> The configuration for High Level consumer is default configuration.
>
> Thanks for your help in advance.
>
> Regards,
> Ankit Jain
>
>
>
> ________________________________
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>