You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by BYEONG-GI KIM <bg...@bluedigm.com> on 2016/01/20 08:56:55 UTC
Kafka 0.9 producer doesn't work
Hello.
I set up the Kafka testbed environment on my VirtualBox, which simply has a
Kafka broker.
I tested the simple consumer & producer scripts, aka
kafka-console-consumer.sh and bin/kafka-console-producer.sh respectively,
and both of them worked fine. I could see the output from the consumer side
whenever typing any words on the producer.
After that, I moved to test a simple java kafka producer/consumer. I copied
and pasted the example source code for producer from
http://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html,
and yeah, unfortunately, it seems not working well; no output was printed
by the above consumer script. There was even no error log on Eclipse.
I really don't know what the problem is... I think that the properties for
both zookeeper and kafka seems fine, since the example scripts worked well,
at least.
I attached my tested source code:
======================================================================
import java.util.Properties;
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.Producer;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.common.KafkaException;
import org.apache.kafka.common.errors.TimeoutException;
public class ProducerExample {
public static void main(String[] args) throws Exception, TimeoutException,
KafkaException {
Properties props = new Properties();
props.put("bootstrap.servers", "10.10.0.40:9092");
props.put("acks", "all");
props.put("retries", 0);
props.put("batch.size", 16384);
// props.put("linger.ms", 1);
props.put("buffer.memory", 33554432);
props.put("key.serializer",
"org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer",
"org.apache.kafka.common.serialization.StringSerializer");
Producer<String, String> producer = new KafkaProducer<String,
String>(props);
try {
for (int i = 0; i < 10; i++) {
producer.send(new ProducerRecord<String, String>("test", 0,
Integer.toString(i), Integer.toString(i)));
}
} catch (TimeoutException te) {
System.out.println(te.getStackTrace());
te.getStackTrace();
} catch (Exception ke) {
System.out.println(ke.getStackTrace());
ke.getStackTrace();
}
producer.close();
}
}
======================================================================
Any advice would really be helpful. Thanks in advance.
Best regards
Kim
Re: Kafka 0.9 producer doesn't work
Posted by BYEONG-GI KIM <bg...@bluedigm.com>.
Thank you for the reply.
I tested it and the result showed timeout.
I guess my kafka setting is something wrong, not the source code.
I'll check what I missed.
Best regards
Kim
2016-01-20 18:03 GMT+09:00 tao xiao <xi...@gmail.com>:
> It is possible that you closed the producer before the messages accumulated
> in batch had been sent out.
>
> You can modify your producer as below to make it a sync call and test
> again.
>
> producer.send(new ProducerRecord<String, String>("test",
> 0, Integer.toString(i), Integer.toString(i))).get();
>
> On Wed, 20 Jan 2016 at 16:31 BYEONG-GI KIM <bg...@bluedigm.com> wrote:
>
> > Sure, I started consumer before starting and sending messages from
> > producer, and my broker version, if you mean the kafka version, is 0.9.0.
> >
> > Best regards
> >
> > Kim
> >
> > 2016-01-20 17:28 GMT+09:00 Steve Tian <st...@gmail.com>:
> >
> > > Did you start your consumer before sending message? Broker version?
> > >
> > > Cheers, Steve
> > >
> > > On Wed, Jan 20, 2016, 3:57 PM BYEONG-GI KIM <bg...@bluedigm.com>
> wrote:
> > >
> > > > Hello.
> > > >
> > > > I set up the Kafka testbed environment on my VirtualBox, which simply
> > > has a
> > > > Kafka broker.
> > > >
> > > > I tested the simple consumer & producer scripts, aka
> > > > kafka-console-consumer.sh and bin/kafka-console-producer.sh
> > respectively,
> > > > and both of them worked fine. I could see the output from the
> consumer
> > > side
> > > > whenever typing any words on the producer.
> > > >
> > > > After that, I moved to test a simple java kafka producer/consumer. I
> > > copied
> > > > and pasted the example source code for producer from
> > > >
> > > >
> > >
> >
> http://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html
> > > > ,
> > > > and yeah, unfortunately, it seems not working well; no output was
> > printed
> > > > by the above consumer script. There was even no error log on Eclipse.
> > > >
> > > > I really don't know what the problem is... I think that the
> properties
> > > for
> > > > both zookeeper and kafka seems fine, since the example scripts worked
> > > well,
> > > > at least.
> > > >
> > > > I attached my tested source code:
> > > >
> ======================================================================
> > > > import java.util.Properties;
> > > >
> > > > import org.apache.kafka.clients.producer.KafkaProducer;
> > > > import org.apache.kafka.clients.producer.Producer;
> > > > import org.apache.kafka.clients.producer.ProducerRecord;
> > > > import org.apache.kafka.common.KafkaException;
> > > > import org.apache.kafka.common.errors.TimeoutException;
> > > >
> > > > public class ProducerExample {
> > > > public static void main(String[] args) throws Exception,
> > > TimeoutException,
> > > > KafkaException {
> > > > Properties props = new Properties();
> > > > props.put("bootstrap.servers", "10.10.0.40:9092");
> > > > props.put("acks", "all");
> > > > props.put("retries", 0);
> > > > props.put("batch.size", 16384);
> > > > // props.put("linger.ms", 1);
> > > > props.put("buffer.memory", 33554432);
> > > > props.put("key.serializer",
> > > > "org.apache.kafka.common.serialization.StringSerializer");
> > > > props.put("value.serializer",
> > > > "org.apache.kafka.common.serialization.StringSerializer");
> > > >
> > > > Producer<String, String> producer = new KafkaProducer<String,
> > > > String>(props);
> > > >
> > > > try {
> > > > for (int i = 0; i < 10; i++) {
> > > > producer.send(new ProducerRecord<String, String>("test", 0,
> > > > Integer.toString(i), Integer.toString(i)));
> > > > }
> > > > } catch (TimeoutException te) {
> > > > System.out.println(te.getStackTrace());
> > > > te.getStackTrace();
> > > > } catch (Exception ke) {
> > > > System.out.println(ke.getStackTrace());
> > > > ke.getStackTrace();
> > > > }
> > > >
> > > > producer.close();
> > > > }
> > > > }
> > > >
> ======================================================================
> > > >
> > > > Any advice would really be helpful. Thanks in advance.
> > > >
> > > > Best regards
> > > >
> > > > Kim
> > > >
> > >
> >
> >
> >
> > --
> > (주)비디 클라우드사업부 와이즈본부 클라우드기술팀 선임
> >
>
--
(주)비디 클라우드사업부 와이즈본부 클라우드기술팀 선임
Re: Kafka 0.9 producer doesn't work
Posted by tao xiao <xi...@gmail.com>.
It is possible that you closed the producer before the messages accumulated
in batch had been sent out.
You can modify your producer as below to make it a sync call and test again.
producer.send(new ProducerRecord<String, String>("test",
0, Integer.toString(i), Integer.toString(i))).get();
On Wed, 20 Jan 2016 at 16:31 BYEONG-GI KIM <bg...@bluedigm.com> wrote:
> Sure, I started consumer before starting and sending messages from
> producer, and my broker version, if you mean the kafka version, is 0.9.0.
>
> Best regards
>
> Kim
>
> 2016-01-20 17:28 GMT+09:00 Steve Tian <st...@gmail.com>:
>
> > Did you start your consumer before sending message? Broker version?
> >
> > Cheers, Steve
> >
> > On Wed, Jan 20, 2016, 3:57 PM BYEONG-GI KIM <bg...@bluedigm.com> wrote:
> >
> > > Hello.
> > >
> > > I set up the Kafka testbed environment on my VirtualBox, which simply
> > has a
> > > Kafka broker.
> > >
> > > I tested the simple consumer & producer scripts, aka
> > > kafka-console-consumer.sh and bin/kafka-console-producer.sh
> respectively,
> > > and both of them worked fine. I could see the output from the consumer
> > side
> > > whenever typing any words on the producer.
> > >
> > > After that, I moved to test a simple java kafka producer/consumer. I
> > copied
> > > and pasted the example source code for producer from
> > >
> > >
> >
> http://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html
> > > ,
> > > and yeah, unfortunately, it seems not working well; no output was
> printed
> > > by the above consumer script. There was even no error log on Eclipse.
> > >
> > > I really don't know what the problem is... I think that the properties
> > for
> > > both zookeeper and kafka seems fine, since the example scripts worked
> > well,
> > > at least.
> > >
> > > I attached my tested source code:
> > > ======================================================================
> > > import java.util.Properties;
> > >
> > > import org.apache.kafka.clients.producer.KafkaProducer;
> > > import org.apache.kafka.clients.producer.Producer;
> > > import org.apache.kafka.clients.producer.ProducerRecord;
> > > import org.apache.kafka.common.KafkaException;
> > > import org.apache.kafka.common.errors.TimeoutException;
> > >
> > > public class ProducerExample {
> > > public static void main(String[] args) throws Exception,
> > TimeoutException,
> > > KafkaException {
> > > Properties props = new Properties();
> > > props.put("bootstrap.servers", "10.10.0.40:9092");
> > > props.put("acks", "all");
> > > props.put("retries", 0);
> > > props.put("batch.size", 16384);
> > > // props.put("linger.ms", 1);
> > > props.put("buffer.memory", 33554432);
> > > props.put("key.serializer",
> > > "org.apache.kafka.common.serialization.StringSerializer");
> > > props.put("value.serializer",
> > > "org.apache.kafka.common.serialization.StringSerializer");
> > >
> > > Producer<String, String> producer = new KafkaProducer<String,
> > > String>(props);
> > >
> > > try {
> > > for (int i = 0; i < 10; i++) {
> > > producer.send(new ProducerRecord<String, String>("test", 0,
> > > Integer.toString(i), Integer.toString(i)));
> > > }
> > > } catch (TimeoutException te) {
> > > System.out.println(te.getStackTrace());
> > > te.getStackTrace();
> > > } catch (Exception ke) {
> > > System.out.println(ke.getStackTrace());
> > > ke.getStackTrace();
> > > }
> > >
> > > producer.close();
> > > }
> > > }
> > > ======================================================================
> > >
> > > Any advice would really be helpful. Thanks in advance.
> > >
> > > Best regards
> > >
> > > Kim
> > >
> >
>
>
>
> --
> (주)비디 클라우드사업부 와이즈본부 클라우드기술팀 선임
>
Re: Kafka 0.9 producer doesn't work
Posted by BYEONG-GI KIM <bg...@bluedigm.com>.
Yes, I opened the kafka-server/zookeeper ports of firewalld.
And I finally noticed what was wrong; I didn't set "advertised.host.name"
property in config/server.properties file... It should be set to the IP of
the VM where the kafka server and zookeeper is running. After I put the IP
of the VM in the property the source code worked very well from my host.
Anyway, thanks indeed for a lot of helps!
Best regards
Kim
2016-01-21 11:48 GMT+09:00 Steve Tian <st...@gmail.com>:
> Have you checked the firewall setting on vm/host?
>
>
> On Thu, Jan 21, 2016, 10:29 AM BYEONG-GI KIM <bg...@bluedigm.com> wrote:
>
>> Hello.
>>
>> I packaged it to an executable jar file and executed it on the VM, and
>> yes, it was successfully worked.
>>
>> I'm really confuse why it didn't work on my Windows10 environment where
>> is on the host environment and worked well on the VM environment... It is
>> weird indeed.
>>
>> Best regards
>>
>> Kim
>>
>> 2016-01-20 18:14 GMT+09:00 Steve Tian <st...@gmail.com>:
>>
>>> Your code works in my environment. Are you able to run your producer
>>> code inside your vm? You can also debug via changing the log level to
>>> DEGUG/TRACE.
>>>
>>> Cheers, Steve
>>>
>>>
>>> On Wed, Jan 20, 2016, 4:30 PM BYEONG-GI KIM <bg...@bluedigm.com> wrote:
>>>
>>>> Sure, I started consumer before starting and sending messages from
>>>> producer, and my broker version, if you mean the kafka version, is 0.9.0.
>>>>
>>>> Best regards
>>>>
>>>> Kim
>>>>
>>>> 2016-01-20 17:28 GMT+09:00 Steve Tian <st...@gmail.com>:
>>>>
>>>>> Did you start your consumer before sending message? Broker version?
>>>>>
>>>>> Cheers, Steve
>>>>>
>>>>> On Wed, Jan 20, 2016, 3:57 PM BYEONG-GI KIM <bg...@bluedigm.com>
>>>>> wrote:
>>>>>
>>>>> > Hello.
>>>>> >
>>>>> > I set up the Kafka testbed environment on my VirtualBox, which
>>>>> simply has a
>>>>> > Kafka broker.
>>>>> >
>>>>> > I tested the simple consumer & producer scripts, aka
>>>>> > kafka-console-consumer.sh and bin/kafka-console-producer.sh
>>>>> respectively,
>>>>> > and both of them worked fine. I could see the output from the
>>>>> consumer side
>>>>> > whenever typing any words on the producer.
>>>>> >
>>>>> > After that, I moved to test a simple java kafka producer/consumer. I
>>>>> copied
>>>>> > and pasted the example source code for producer from
>>>>> >
>>>>> >
>>>>> http://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html
>>>>> > ,
>>>>> > and yeah, unfortunately, it seems not working well; no output was
>>>>> printed
>>>>> > by the above consumer script. There was even no error log on Eclipse.
>>>>> >
>>>>> > I really don't know what the problem is... I think that the
>>>>> properties for
>>>>> > both zookeeper and kafka seems fine, since the example scripts
>>>>> worked well,
>>>>> > at least.
>>>>> >
>>>>> > I attached my tested source code:
>>>>> >
>>>>> ======================================================================
>>>>> > import java.util.Properties;
>>>>> >
>>>>> > import org.apache.kafka.clients.producer.KafkaProducer;
>>>>> > import org.apache.kafka.clients.producer.Producer;
>>>>> > import org.apache.kafka.clients.producer.ProducerRecord;
>>>>> > import org.apache.kafka.common.KafkaException;
>>>>> > import org.apache.kafka.common.errors.TimeoutException;
>>>>> >
>>>>> > public class ProducerExample {
>>>>> > public static void main(String[] args) throws Exception,
>>>>> TimeoutException,
>>>>> > KafkaException {
>>>>> > Properties props = new Properties();
>>>>> > props.put("bootstrap.servers", "10.10.0.40:9092");
>>>>> > props.put("acks", "all");
>>>>> > props.put("retries", 0);
>>>>> > props.put("batch.size", 16384);
>>>>> > // props.put("linger.ms", 1);
>>>>> > props.put("buffer.memory", 33554432);
>>>>> > props.put("key.serializer",
>>>>> > "org.apache.kafka.common.serialization.StringSerializer");
>>>>> > props.put("value.serializer",
>>>>> > "org.apache.kafka.common.serialization.StringSerializer");
>>>>> >
>>>>> > Producer<String, String> producer = new KafkaProducer<String,
>>>>> > String>(props);
>>>>> >
>>>>> > try {
>>>>> > for (int i = 0; i < 10; i++) {
>>>>> > producer.send(new ProducerRecord<String, String>("test", 0,
>>>>> > Integer.toString(i), Integer.toString(i)));
>>>>> > }
>>>>> > } catch (TimeoutException te) {
>>>>> > System.out.println(te.getStackTrace());
>>>>> > te.getStackTrace();
>>>>> > } catch (Exception ke) {
>>>>> > System.out.println(ke.getStackTrace());
>>>>> > ke.getStackTrace();
>>>>> > }
>>>>> >
>>>>> > producer.close();
>>>>> > }
>>>>> > }
>>>>> >
>>>>> ======================================================================
>>>>> >
>>>>> > Any advice would really be helpful. Thanks in advance.
>>>>> >
>>>>> > Best regards
>>>>> >
>>>>> > Kim
>>>>> >
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> (주)비디 클라우드사업부 와이즈본부 클라우드기술팀 선임
>>>>
>>>
>>
>>
>> --
>> (주)비디 클라우드사업부 와이즈본부 클라우드기술팀 선임
>>
>
--
(주)비디 클라우드사업부 와이즈본부 클라우드기술팀 선임
Re: Kafka 0.9 producer doesn't work
Posted by Steve Tian <st...@gmail.com>.
Have you checked the firewall setting on vm/host?
On Thu, Jan 21, 2016, 10:29 AM BYEONG-GI KIM <bg...@bluedigm.com> wrote:
> Hello.
>
> I packaged it to an executable jar file and executed it on the VM, and
> yes, it was successfully worked.
>
> I'm really confuse why it didn't work on my Windows10 environment where is
> on the host environment and worked well on the VM environment... It is
> weird indeed.
>
> Best regards
>
> Kim
>
> 2016-01-20 18:14 GMT+09:00 Steve Tian <st...@gmail.com>:
>
>> Your code works in my environment. Are you able to run your producer
>> code inside your vm? You can also debug via changing the log level to
>> DEGUG/TRACE.
>>
>> Cheers, Steve
>>
>>
>> On Wed, Jan 20, 2016, 4:30 PM BYEONG-GI KIM <bg...@bluedigm.com> wrote:
>>
>>> Sure, I started consumer before starting and sending messages from
>>> producer, and my broker version, if you mean the kafka version, is 0.9.0.
>>>
>>> Best regards
>>>
>>> Kim
>>>
>>> 2016-01-20 17:28 GMT+09:00 Steve Tian <st...@gmail.com>:
>>>
>>>> Did you start your consumer before sending message? Broker version?
>>>>
>>>> Cheers, Steve
>>>>
>>>> On Wed, Jan 20, 2016, 3:57 PM BYEONG-GI KIM <bg...@bluedigm.com> wrote:
>>>>
>>>> > Hello.
>>>> >
>>>> > I set up the Kafka testbed environment on my VirtualBox, which simply
>>>> has a
>>>> > Kafka broker.
>>>> >
>>>> > I tested the simple consumer & producer scripts, aka
>>>> > kafka-console-consumer.sh and bin/kafka-console-producer.sh
>>>> respectively,
>>>> > and both of them worked fine. I could see the output from the
>>>> consumer side
>>>> > whenever typing any words on the producer.
>>>> >
>>>> > After that, I moved to test a simple java kafka producer/consumer. I
>>>> copied
>>>> > and pasted the example source code for producer from
>>>> >
>>>> >
>>>> http://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html
>>>> > ,
>>>> > and yeah, unfortunately, it seems not working well; no output was
>>>> printed
>>>> > by the above consumer script. There was even no error log on Eclipse.
>>>> >
>>>> > I really don't know what the problem is... I think that the
>>>> properties for
>>>> > both zookeeper and kafka seems fine, since the example scripts worked
>>>> well,
>>>> > at least.
>>>> >
>>>> > I attached my tested source code:
>>>> > ======================================================================
>>>> > import java.util.Properties;
>>>> >
>>>> > import org.apache.kafka.clients.producer.KafkaProducer;
>>>> > import org.apache.kafka.clients.producer.Producer;
>>>> > import org.apache.kafka.clients.producer.ProducerRecord;
>>>> > import org.apache.kafka.common.KafkaException;
>>>> > import org.apache.kafka.common.errors.TimeoutException;
>>>> >
>>>> > public class ProducerExample {
>>>> > public static void main(String[] args) throws Exception,
>>>> TimeoutException,
>>>> > KafkaException {
>>>> > Properties props = new Properties();
>>>> > props.put("bootstrap.servers", "10.10.0.40:9092");
>>>> > props.put("acks", "all");
>>>> > props.put("retries", 0);
>>>> > props.put("batch.size", 16384);
>>>> > // props.put("linger.ms", 1);
>>>> > props.put("buffer.memory", 33554432);
>>>> > props.put("key.serializer",
>>>> > "org.apache.kafka.common.serialization.StringSerializer");
>>>> > props.put("value.serializer",
>>>> > "org.apache.kafka.common.serialization.StringSerializer");
>>>> >
>>>> > Producer<String, String> producer = new KafkaProducer<String,
>>>> > String>(props);
>>>> >
>>>> > try {
>>>> > for (int i = 0; i < 10; i++) {
>>>> > producer.send(new ProducerRecord<String, String>("test", 0,
>>>> > Integer.toString(i), Integer.toString(i)));
>>>> > }
>>>> > } catch (TimeoutException te) {
>>>> > System.out.println(te.getStackTrace());
>>>> > te.getStackTrace();
>>>> > } catch (Exception ke) {
>>>> > System.out.println(ke.getStackTrace());
>>>> > ke.getStackTrace();
>>>> > }
>>>> >
>>>> > producer.close();
>>>> > }
>>>> > }
>>>> > ======================================================================
>>>> >
>>>> > Any advice would really be helpful. Thanks in advance.
>>>> >
>>>> > Best regards
>>>> >
>>>> > Kim
>>>> >
>>>>
>>>
>>>
>>>
>>> --
>>> (주)비디 클라우드사업부 와이즈본부 클라우드기술팀 선임
>>>
>>
>
>
> --
> (주)비디 클라우드사업부 와이즈본부 클라우드기술팀 선임
>
Re: Kafka 0.9 producer doesn't work
Posted by BYEONG-GI KIM <bg...@bluedigm.com>.
Hello.
I packaged it to an executable jar file and executed it on the VM, and yes,
it was successfully worked.
I'm really confuse why it didn't work on my Windows10 environment where is
on the host environment and worked well on the VM environment... It is
weird indeed.
Best regards
Kim
2016-01-20 18:14 GMT+09:00 Steve Tian <st...@gmail.com>:
> Your code works in my environment. Are you able to run your producer code
> inside your vm? You can also debug via changing the log level to
> DEGUG/TRACE.
>
> Cheers, Steve
>
>
> On Wed, Jan 20, 2016, 4:30 PM BYEONG-GI KIM <bg...@bluedigm.com> wrote:
>
>> Sure, I started consumer before starting and sending messages from
>> producer, and my broker version, if you mean the kafka version, is 0.9.0.
>>
>> Best regards
>>
>> Kim
>>
>> 2016-01-20 17:28 GMT+09:00 Steve Tian <st...@gmail.com>:
>>
>>> Did you start your consumer before sending message? Broker version?
>>>
>>> Cheers, Steve
>>>
>>> On Wed, Jan 20, 2016, 3:57 PM BYEONG-GI KIM <bg...@bluedigm.com> wrote:
>>>
>>> > Hello.
>>> >
>>> > I set up the Kafka testbed environment on my VirtualBox, which simply
>>> has a
>>> > Kafka broker.
>>> >
>>> > I tested the simple consumer & producer scripts, aka
>>> > kafka-console-consumer.sh and bin/kafka-console-producer.sh
>>> respectively,
>>> > and both of them worked fine. I could see the output from the consumer
>>> side
>>> > whenever typing any words on the producer.
>>> >
>>> > After that, I moved to test a simple java kafka producer/consumer. I
>>> copied
>>> > and pasted the example source code for producer from
>>> >
>>> >
>>> http://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html
>>> > ,
>>> > and yeah, unfortunately, it seems not working well; no output was
>>> printed
>>> > by the above consumer script. There was even no error log on Eclipse.
>>> >
>>> > I really don't know what the problem is... I think that the properties
>>> for
>>> > both zookeeper and kafka seems fine, since the example scripts worked
>>> well,
>>> > at least.
>>> >
>>> > I attached my tested source code:
>>> > ======================================================================
>>> > import java.util.Properties;
>>> >
>>> > import org.apache.kafka.clients.producer.KafkaProducer;
>>> > import org.apache.kafka.clients.producer.Producer;
>>> > import org.apache.kafka.clients.producer.ProducerRecord;
>>> > import org.apache.kafka.common.KafkaException;
>>> > import org.apache.kafka.common.errors.TimeoutException;
>>> >
>>> > public class ProducerExample {
>>> > public static void main(String[] args) throws Exception,
>>> TimeoutException,
>>> > KafkaException {
>>> > Properties props = new Properties();
>>> > props.put("bootstrap.servers", "10.10.0.40:9092");
>>> > props.put("acks", "all");
>>> > props.put("retries", 0);
>>> > props.put("batch.size", 16384);
>>> > // props.put("linger.ms", 1);
>>> > props.put("buffer.memory", 33554432);
>>> > props.put("key.serializer",
>>> > "org.apache.kafka.common.serialization.StringSerializer");
>>> > props.put("value.serializer",
>>> > "org.apache.kafka.common.serialization.StringSerializer");
>>> >
>>> > Producer<String, String> producer = new KafkaProducer<String,
>>> > String>(props);
>>> >
>>> > try {
>>> > for (int i = 0; i < 10; i++) {
>>> > producer.send(new ProducerRecord<String, String>("test", 0,
>>> > Integer.toString(i), Integer.toString(i)));
>>> > }
>>> > } catch (TimeoutException te) {
>>> > System.out.println(te.getStackTrace());
>>> > te.getStackTrace();
>>> > } catch (Exception ke) {
>>> > System.out.println(ke.getStackTrace());
>>> > ke.getStackTrace();
>>> > }
>>> >
>>> > producer.close();
>>> > }
>>> > }
>>> > ======================================================================
>>> >
>>> > Any advice would really be helpful. Thanks in advance.
>>> >
>>> > Best regards
>>> >
>>> > Kim
>>> >
>>>
>>
>>
>>
>> --
>> (주)비디 클라우드사업부 와이즈본부 클라우드기술팀 선임
>>
>
--
(주)비디 클라우드사업부 와이즈본부 클라우드기술팀 선임
Re: Kafka 0.9 producer doesn't work
Posted by Steve Tian <st...@gmail.com>.
Yes, that's the version I was using.
If all you need is Java client, then you can try:
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>0.9.0.0</version>
</dependency>
Cheers, Steve
On Thu, Jan 21, 2016, 9:04 AM BYEONG-GI KIM <bg...@bluedigm.com> wrote:
> Dear Steve
>
> Could you tell me what kafka version you are using for the source code's
> package?
>
> I included the kafka library from maven repository (
> http://mvnrepository.com/artifact/org.apache.kafka), and the artifactId
> is kafka_2.11 and version is 0.9.0.0. The link is as below:
>
> http://mvnrepository.com/artifact/org.apache.kafka/kafka_2.11/0.9.0.0
>
> The maven dependency is as below:
>
> <dependency>
> <groupId>org.apache.kafka</groupId>
> <artifactId>kafka_2.11</artifactId>
> <version>0.9.0.0</version>
> </dependency>
>
> Are you using this version?
>
> Best regards
>
> Kim
>
> 2016-01-20 18:14 GMT+09:00 Steve Tian <st...@gmail.com>:
>
>> Your code works in my environment. Are you able to run your producer
>> code inside your vm? You can also debug via changing the log level to
>> DEGUG/TRACE.
>>
>> Cheers, Steve
>>
>>
>> On Wed, Jan 20, 2016, 4:30 PM BYEONG-GI KIM <bg...@bluedigm.com> wrote:
>>
>>> Sure, I started consumer before starting and sending messages from
>>> producer, and my broker version, if you mean the kafka version, is 0.9.0.
>>>
>>> Best regards
>>>
>>> Kim
>>>
>>> 2016-01-20 17:28 GMT+09:00 Steve Tian <st...@gmail.com>:
>>>
>>>> Did you start your consumer before sending message? Broker version?
>>>>
>>>> Cheers, Steve
>>>>
>>>> On Wed, Jan 20, 2016, 3:57 PM BYEONG-GI KIM <bg...@bluedigm.com> wrote:
>>>>
>>>> > Hello.
>>>> >
>>>> > I set up the Kafka testbed environment on my VirtualBox, which simply
>>>> has a
>>>> > Kafka broker.
>>>> >
>>>> > I tested the simple consumer & producer scripts, aka
>>>> > kafka-console-consumer.sh and bin/kafka-console-producer.sh
>>>> respectively,
>>>> > and both of them worked fine. I could see the output from the
>>>> consumer side
>>>> > whenever typing any words on the producer.
>>>> >
>>>> > After that, I moved to test a simple java kafka producer/consumer. I
>>>> copied
>>>> > and pasted the example source code for producer from
>>>> >
>>>> >
>>>> http://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html
>>>> > ,
>>>> > and yeah, unfortunately, it seems not working well; no output was
>>>> printed
>>>> > by the above consumer script. There was even no error log on Eclipse.
>>>> >
>>>> > I really don't know what the problem is... I think that the
>>>> properties for
>>>> > both zookeeper and kafka seems fine, since the example scripts worked
>>>> well,
>>>> > at least.
>>>> >
>>>> > I attached my tested source code:
>>>> > ======================================================================
>>>> > import java.util.Properties;
>>>> >
>>>> > import org.apache.kafka.clients.producer.KafkaProducer;
>>>> > import org.apache.kafka.clients.producer.Producer;
>>>> > import org.apache.kafka.clients.producer.ProducerRecord;
>>>> > import org.apache.kafka.common.KafkaException;
>>>> > import org.apache.kafka.common.errors.TimeoutException;
>>>> >
>>>> > public class ProducerExample {
>>>> > public static void main(String[] args) throws Exception,
>>>> TimeoutException,
>>>> > KafkaException {
>>>> > Properties props = new Properties();
>>>> > props.put("bootstrap.servers", "10.10.0.40:9092");
>>>> > props.put("acks", "all");
>>>> > props.put("retries", 0);
>>>> > props.put("batch.size", 16384);
>>>> > // props.put("linger.ms", 1);
>>>> > props.put("buffer.memory", 33554432);
>>>> > props.put("key.serializer",
>>>> > "org.apache.kafka.common.serialization.StringSerializer");
>>>> > props.put("value.serializer",
>>>> > "org.apache.kafka.common.serialization.StringSerializer");
>>>> >
>>>> > Producer<String, String> producer = new KafkaProducer<String,
>>>> > String>(props);
>>>> >
>>>> > try {
>>>> > for (int i = 0; i < 10; i++) {
>>>> > producer.send(new ProducerRecord<String, String>("test", 0,
>>>> > Integer.toString(i), Integer.toString(i)));
>>>> > }
>>>> > } catch (TimeoutException te) {
>>>> > System.out.println(te.getStackTrace());
>>>> > te.getStackTrace();
>>>> > } catch (Exception ke) {
>>>> > System.out.println(ke.getStackTrace());
>>>> > ke.getStackTrace();
>>>> > }
>>>> >
>>>> > producer.close();
>>>> > }
>>>> > }
>>>> > ======================================================================
>>>> >
>>>> > Any advice would really be helpful. Thanks in advance.
>>>> >
>>>> > Best regards
>>>> >
>>>> > Kim
>>>> >
>>>>
>>>
>>>
>>>
>>> --
>>> (주)비디 클라우드사업부 와이즈본부 클라우드기술팀 선임
>>>
>>
>
>
> --
> (주)비디 클라우드사업부 와이즈본부 클라우드기술팀 선임
>
Re: Kafka 0.9 producer doesn't work
Posted by BYEONG-GI KIM <bg...@bluedigm.com>.
Dear Steve
Could you tell me what kafka version you are using for the source code's
package?
I included the kafka library from maven repository (
http://mvnrepository.com/artifact/org.apache.kafka), and the artifactId is
kafka_2.11 and version is 0.9.0.0. The link is as below:
http://mvnrepository.com/artifact/org.apache.kafka/kafka_2.11/0.9.0.0
The maven dependency is as below:
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.11</artifactId>
<version>0.9.0.0</version>
</dependency>
Are you using this version?
Best regards
Kim
2016-01-20 18:14 GMT+09:00 Steve Tian <st...@gmail.com>:
> Your code works in my environment. Are you able to run your producer code
> inside your vm? You can also debug via changing the log level to
> DEGUG/TRACE.
>
> Cheers, Steve
>
>
> On Wed, Jan 20, 2016, 4:30 PM BYEONG-GI KIM <bg...@bluedigm.com> wrote:
>
>> Sure, I started consumer before starting and sending messages from
>> producer, and my broker version, if you mean the kafka version, is 0.9.0.
>>
>> Best regards
>>
>> Kim
>>
>> 2016-01-20 17:28 GMT+09:00 Steve Tian <st...@gmail.com>:
>>
>>> Did you start your consumer before sending message? Broker version?
>>>
>>> Cheers, Steve
>>>
>>> On Wed, Jan 20, 2016, 3:57 PM BYEONG-GI KIM <bg...@bluedigm.com> wrote:
>>>
>>> > Hello.
>>> >
>>> > I set up the Kafka testbed environment on my VirtualBox, which simply
>>> has a
>>> > Kafka broker.
>>> >
>>> > I tested the simple consumer & producer scripts, aka
>>> > kafka-console-consumer.sh and bin/kafka-console-producer.sh
>>> respectively,
>>> > and both of them worked fine. I could see the output from the consumer
>>> side
>>> > whenever typing any words on the producer.
>>> >
>>> > After that, I moved to test a simple java kafka producer/consumer. I
>>> copied
>>> > and pasted the example source code for producer from
>>> >
>>> >
>>> http://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html
>>> > ,
>>> > and yeah, unfortunately, it seems not working well; no output was
>>> printed
>>> > by the above consumer script. There was even no error log on Eclipse.
>>> >
>>> > I really don't know what the problem is... I think that the properties
>>> for
>>> > both zookeeper and kafka seems fine, since the example scripts worked
>>> well,
>>> > at least.
>>> >
>>> > I attached my tested source code:
>>> > ======================================================================
>>> > import java.util.Properties;
>>> >
>>> > import org.apache.kafka.clients.producer.KafkaProducer;
>>> > import org.apache.kafka.clients.producer.Producer;
>>> > import org.apache.kafka.clients.producer.ProducerRecord;
>>> > import org.apache.kafka.common.KafkaException;
>>> > import org.apache.kafka.common.errors.TimeoutException;
>>> >
>>> > public class ProducerExample {
>>> > public static void main(String[] args) throws Exception,
>>> TimeoutException,
>>> > KafkaException {
>>> > Properties props = new Properties();
>>> > props.put("bootstrap.servers", "10.10.0.40:9092");
>>> > props.put("acks", "all");
>>> > props.put("retries", 0);
>>> > props.put("batch.size", 16384);
>>> > // props.put("linger.ms", 1);
>>> > props.put("buffer.memory", 33554432);
>>> > props.put("key.serializer",
>>> > "org.apache.kafka.common.serialization.StringSerializer");
>>> > props.put("value.serializer",
>>> > "org.apache.kafka.common.serialization.StringSerializer");
>>> >
>>> > Producer<String, String> producer = new KafkaProducer<String,
>>> > String>(props);
>>> >
>>> > try {
>>> > for (int i = 0; i < 10; i++) {
>>> > producer.send(new ProducerRecord<String, String>("test", 0,
>>> > Integer.toString(i), Integer.toString(i)));
>>> > }
>>> > } catch (TimeoutException te) {
>>> > System.out.println(te.getStackTrace());
>>> > te.getStackTrace();
>>> > } catch (Exception ke) {
>>> > System.out.println(ke.getStackTrace());
>>> > ke.getStackTrace();
>>> > }
>>> >
>>> > producer.close();
>>> > }
>>> > }
>>> > ======================================================================
>>> >
>>> > Any advice would really be helpful. Thanks in advance.
>>> >
>>> > Best regards
>>> >
>>> > Kim
>>> >
>>>
>>
>>
>>
>> --
>> (주)비디 클라우드사업부 와이즈본부 클라우드기술팀 선임
>>
>
--
(주)비디 클라우드사업부 와이즈본부 클라우드기술팀 선임
Re: Kafka 0.9 producer doesn't work
Posted by Steve Tian <st...@gmail.com>.
Your code works in my environment. Are you able to run your producer code
inside your vm? You can also debug via changing the log level to
DEGUG/TRACE.
Cheers, Steve
On Wed, Jan 20, 2016, 4:30 PM BYEONG-GI KIM <bg...@bluedigm.com> wrote:
> Sure, I started consumer before starting and sending messages from
> producer, and my broker version, if you mean the kafka version, is 0.9.0.
>
> Best regards
>
> Kim
>
> 2016-01-20 17:28 GMT+09:00 Steve Tian <st...@gmail.com>:
>
>> Did you start your consumer before sending message? Broker version?
>>
>> Cheers, Steve
>>
>> On Wed, Jan 20, 2016, 3:57 PM BYEONG-GI KIM <bg...@bluedigm.com> wrote:
>>
>> > Hello.
>> >
>> > I set up the Kafka testbed environment on my VirtualBox, which simply
>> has a
>> > Kafka broker.
>> >
>> > I tested the simple consumer & producer scripts, aka
>> > kafka-console-consumer.sh and bin/kafka-console-producer.sh
>> respectively,
>> > and both of them worked fine. I could see the output from the consumer
>> side
>> > whenever typing any words on the producer.
>> >
>> > After that, I moved to test a simple java kafka producer/consumer. I
>> copied
>> > and pasted the example source code for producer from
>> >
>> >
>> http://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html
>> > ,
>> > and yeah, unfortunately, it seems not working well; no output was
>> printed
>> > by the above consumer script. There was even no error log on Eclipse.
>> >
>> > I really don't know what the problem is... I think that the properties
>> for
>> > both zookeeper and kafka seems fine, since the example scripts worked
>> well,
>> > at least.
>> >
>> > I attached my tested source code:
>> > ======================================================================
>> > import java.util.Properties;
>> >
>> > import org.apache.kafka.clients.producer.KafkaProducer;
>> > import org.apache.kafka.clients.producer.Producer;
>> > import org.apache.kafka.clients.producer.ProducerRecord;
>> > import org.apache.kafka.common.KafkaException;
>> > import org.apache.kafka.common.errors.TimeoutException;
>> >
>> > public class ProducerExample {
>> > public static void main(String[] args) throws Exception,
>> TimeoutException,
>> > KafkaException {
>> > Properties props = new Properties();
>> > props.put("bootstrap.servers", "10.10.0.40:9092");
>> > props.put("acks", "all");
>> > props.put("retries", 0);
>> > props.put("batch.size", 16384);
>> > // props.put("linger.ms", 1);
>> > props.put("buffer.memory", 33554432);
>> > props.put("key.serializer",
>> > "org.apache.kafka.common.serialization.StringSerializer");
>> > props.put("value.serializer",
>> > "org.apache.kafka.common.serialization.StringSerializer");
>> >
>> > Producer<String, String> producer = new KafkaProducer<String,
>> > String>(props);
>> >
>> > try {
>> > for (int i = 0; i < 10; i++) {
>> > producer.send(new ProducerRecord<String, String>("test", 0,
>> > Integer.toString(i), Integer.toString(i)));
>> > }
>> > } catch (TimeoutException te) {
>> > System.out.println(te.getStackTrace());
>> > te.getStackTrace();
>> > } catch (Exception ke) {
>> > System.out.println(ke.getStackTrace());
>> > ke.getStackTrace();
>> > }
>> >
>> > producer.close();
>> > }
>> > }
>> > ======================================================================
>> >
>> > Any advice would really be helpful. Thanks in advance.
>> >
>> > Best regards
>> >
>> > Kim
>> >
>>
>
>
>
> --
> (주)비디 클라우드사업부 와이즈본부 클라우드기술팀 선임
>
Re: Kafka 0.9 producer doesn't work
Posted by BYEONG-GI KIM <bg...@bluedigm.com>.
Sure, I started consumer before starting and sending messages from
producer, and my broker version, if you mean the kafka version, is 0.9.0.
Best regards
Kim
2016-01-20 17:28 GMT+09:00 Steve Tian <st...@gmail.com>:
> Did you start your consumer before sending message? Broker version?
>
> Cheers, Steve
>
> On Wed, Jan 20, 2016, 3:57 PM BYEONG-GI KIM <bg...@bluedigm.com> wrote:
>
> > Hello.
> >
> > I set up the Kafka testbed environment on my VirtualBox, which simply
> has a
> > Kafka broker.
> >
> > I tested the simple consumer & producer scripts, aka
> > kafka-console-consumer.sh and bin/kafka-console-producer.sh respectively,
> > and both of them worked fine. I could see the output from the consumer
> side
> > whenever typing any words on the producer.
> >
> > After that, I moved to test a simple java kafka producer/consumer. I
> copied
> > and pasted the example source code for producer from
> >
> >
> http://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html
> > ,
> > and yeah, unfortunately, it seems not working well; no output was printed
> > by the above consumer script. There was even no error log on Eclipse.
> >
> > I really don't know what the problem is... I think that the properties
> for
> > both zookeeper and kafka seems fine, since the example scripts worked
> well,
> > at least.
> >
> > I attached my tested source code:
> > ======================================================================
> > import java.util.Properties;
> >
> > import org.apache.kafka.clients.producer.KafkaProducer;
> > import org.apache.kafka.clients.producer.Producer;
> > import org.apache.kafka.clients.producer.ProducerRecord;
> > import org.apache.kafka.common.KafkaException;
> > import org.apache.kafka.common.errors.TimeoutException;
> >
> > public class ProducerExample {
> > public static void main(String[] args) throws Exception,
> TimeoutException,
> > KafkaException {
> > Properties props = new Properties();
> > props.put("bootstrap.servers", "10.10.0.40:9092");
> > props.put("acks", "all");
> > props.put("retries", 0);
> > props.put("batch.size", 16384);
> > // props.put("linger.ms", 1);
> > props.put("buffer.memory", 33554432);
> > props.put("key.serializer",
> > "org.apache.kafka.common.serialization.StringSerializer");
> > props.put("value.serializer",
> > "org.apache.kafka.common.serialization.StringSerializer");
> >
> > Producer<String, String> producer = new KafkaProducer<String,
> > String>(props);
> >
> > try {
> > for (int i = 0; i < 10; i++) {
> > producer.send(new ProducerRecord<String, String>("test", 0,
> > Integer.toString(i), Integer.toString(i)));
> > }
> > } catch (TimeoutException te) {
> > System.out.println(te.getStackTrace());
> > te.getStackTrace();
> > } catch (Exception ke) {
> > System.out.println(ke.getStackTrace());
> > ke.getStackTrace();
> > }
> >
> > producer.close();
> > }
> > }
> > ======================================================================
> >
> > Any advice would really be helpful. Thanks in advance.
> >
> > Best regards
> >
> > Kim
> >
>
--
(주)비디 클라우드사업부 와이즈본부 클라우드기술팀 선임
Re: Kafka 0.9 producer doesn't work
Posted by Steve Tian <st...@gmail.com>.
Did you start your consumer before sending message? Broker version?
Cheers, Steve
On Wed, Jan 20, 2016, 3:57 PM BYEONG-GI KIM <bg...@bluedigm.com> wrote:
> Hello.
>
> I set up the Kafka testbed environment on my VirtualBox, which simply has a
> Kafka broker.
>
> I tested the simple consumer & producer scripts, aka
> kafka-console-consumer.sh and bin/kafka-console-producer.sh respectively,
> and both of them worked fine. I could see the output from the consumer side
> whenever typing any words on the producer.
>
> After that, I moved to test a simple java kafka producer/consumer. I copied
> and pasted the example source code for producer from
>
> http://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html
> ,
> and yeah, unfortunately, it seems not working well; no output was printed
> by the above consumer script. There was even no error log on Eclipse.
>
> I really don't know what the problem is... I think that the properties for
> both zookeeper and kafka seems fine, since the example scripts worked well,
> at least.
>
> I attached my tested source code:
> ======================================================================
> import java.util.Properties;
>
> import org.apache.kafka.clients.producer.KafkaProducer;
> import org.apache.kafka.clients.producer.Producer;
> import org.apache.kafka.clients.producer.ProducerRecord;
> import org.apache.kafka.common.KafkaException;
> import org.apache.kafka.common.errors.TimeoutException;
>
> public class ProducerExample {
> public static void main(String[] args) throws Exception, TimeoutException,
> KafkaException {
> Properties props = new Properties();
> props.put("bootstrap.servers", "10.10.0.40:9092");
> props.put("acks", "all");
> props.put("retries", 0);
> props.put("batch.size", 16384);
> // props.put("linger.ms", 1);
> props.put("buffer.memory", 33554432);
> props.put("key.serializer",
> "org.apache.kafka.common.serialization.StringSerializer");
> props.put("value.serializer",
> "org.apache.kafka.common.serialization.StringSerializer");
>
> Producer<String, String> producer = new KafkaProducer<String,
> String>(props);
>
> try {
> for (int i = 0; i < 10; i++) {
> producer.send(new ProducerRecord<String, String>("test", 0,
> Integer.toString(i), Integer.toString(i)));
> }
> } catch (TimeoutException te) {
> System.out.println(te.getStackTrace());
> te.getStackTrace();
> } catch (Exception ke) {
> System.out.println(ke.getStackTrace());
> ke.getStackTrace();
> }
>
> producer.close();
> }
> }
> ======================================================================
>
> Any advice would really be helpful. Thanks in advance.
>
> Best regards
>
> Kim
>