Kafka Consumer Example source code seems not working correctly

[BYEONG-GI KIM <bg...@bluedigm.com>]

Hello. I'm a new who are getting started to learn Kafka.

I'm now trying to develop a sample java source code to test the new Kafka
Consumer, but it seems not working correctly. I've been testing both of
Kafka Consumer versions; the old one called as High Level Consumer and
another one called as a new Consumer which has been added at 0.9.0 version.

The problem is that both of those example codes seems not be able to pull
any messages from a kafka broker, while a sample Producer,
kafka-console-producer.sh in kafka source package, is running.

In detail, for example, when the Consumer Group Example source code
described in
https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example
was tested, it was automatically shutting down thread(s) after a while. I
guess that the run() in ConsumerTest.java seems not working correctly,
especially the "while" parts, because it did not show any println outputs
that must show subscribed messages.

Any advice and hint would greatly be helpful for me. Thanks in advance!

Best regards

bgkim

Re: Kafka Consumer Example source code seems not working correctly

[BYEONG-GI KIM <bg...@bluedigm.com>]

Hello.

I've been trying to debug the source code and I noticed that the execution
of source code was stopped at the lane "ConsumerRecords<String, String>
records = consumer.poll(100);". Furthermore, I also tested ProducerExample,
but it also seems not working correctly, at least the source code generated
messages and send these to kafka server via "producer.send(new
ProducerRecord<String, String>("test", "Hello!"));". However, the
"bin/kafka-console-consumer.sh" could not receive any message from the
producer.

It is really weird for me... What am I doing wrong? I think I set
topic/ip/port properly, at least.

Thanks in advance.

Best regards

KIM

2016-01-19 11:20 GMT+09:00 BYEONG-GI KIM <bg...@bluedigm.com>:

> Hello. I'm a new who are getting started to learn Kafka.
>
> I'm now trying to develop a sample java source code to test the new Kafka
> Consumer, but it seems not working correctly. I've been testing both of
> Kafka Consumer versions; the old one called as High Level Consumer and
> another one called as a new Consumer which has been added at 0.9.0 version.
>
> The problem is that both of those example codes seems not be able to pull
> any messages from a kafka broker, while a sample Producer,
> kafka-console-producer.sh in kafka source package, is running.
>
> In detail, for example, when the Consumer Group Example source code
> described in
> https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example
> was tested, it was automatically shutting down thread(s) after a while. I
> guess that the run() in ConsumerTest.java seems not working correctly,
> especially the "while" parts, because it did not show any println outputs
> that must show subscribed messages.
>
> Any advice and hint would greatly be helpful for me. Thanks in advance!
>
> Best regards
>
> bgkim
>
>
>
>


-- 
(주)비디 클라우드사업부 와이즈본부 클라우드기술팀 선임