You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Hardik Pandya <sm...@gmail.com> on 2014/11/08 22:25:36 UTC
Issues Running Kafka Producer Java example
Hello Champs,
I am trying to run first java producer example.
Upon running this example, producer successfully sends the message, at
least it looks like it does, there is no java dump
But trying to verify the messages on consumer side - it does not return any
data sent by producer
https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+Producer+Example
here's my Producer code - thanks in advance!
package example.kafka;
import java.util.Date;
import java.util.Properties;
import java.util.Random;
import kafka.javaapi.producer.Producer;
import kafka.producer.KeyedMessage;
import kafka.producer.ProducerConfig;
public class TestProducer {
/**
* @param args
*/
public static void main(String[] args) {
// TODO Auto-generated method stub
long events = Long.parseLong(args[0]);
Properties props = new Properties();
props.put("metadata.broker.list", "10.0.2.15:9092,10.0.2.15:9093,
10.0.2.15:9094,10.0.2.15:9095");
props.put("serializer.class", "kafka.serializer.StringEncoder");
props.put("partitioner.class", "example.kafka.SimplePartitioner");
props.put("request.required.acks", "1");
ProducerConfig config = new ProducerConfig(props);
Producer<String, String> producer = new Producer<String, String>(config);
Random rnd = new Random();
for(long event=0;event < events;event++) {
Long runtime = new Date().getTime();
String msgKey= "192.168.2." + rnd.nextInt(255);
String msg = runtime + ",www.exmaple.com," + msgKey;
KeyedMessage<String, String> data = new KeyedMessage<String,
String>("page_visits", msgKey, msg);
producer.send(data);
System.out.println("message sent");
}
producer.close();
}
}
Re: Issues Running Kafka Producer Java example
Posted by Gwen Shapira <gs...@cloudera.com>.
The producer code here looks fine. It may be an issue with the consumer, or
how the consumer is used.
If you are running the producer before starting a consumer, make sure you
get all messages by setting auto.offset.reset=smallest (in the console
consumer you can use --from-beginning)
Also, you can use the ConsumerOffsetChecker tool to see:
1. How much data you have in the topic
2. Whether the consumer is consuming anything at all
Gwen
On Sat, Nov 8, 2014 at 1:25 PM, Hardik Pandya <sm...@gmail.com>
wrote:
> Hello Champs,
>
> I am trying to run first java producer example.
>
> Upon running this example, producer successfully sends the message, at
> least it looks like it does, there is no java dump
>
> But trying to verify the messages on consumer side - it does not return any
> data sent by producer
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+Producer+Example
>
> here's my Producer code - thanks in advance!
>
> package example.kafka;
>
> import java.util.Date;
> import java.util.Properties;
> import java.util.Random;
>
> import kafka.javaapi.producer.Producer;
> import kafka.producer.KeyedMessage;
> import kafka.producer.ProducerConfig;
>
> public class TestProducer {
>
> /**
> * @param args
> */
> public static void main(String[] args) {
> // TODO Auto-generated method stub
> long events = Long.parseLong(args[0]);
> Properties props = new Properties();
> props.put("metadata.broker.list", "10.0.2.15:9092,10.0.2.15:9093,
> 10.0.2.15:9094,10.0.2.15:9095");
> props.put("serializer.class", "kafka.serializer.StringEncoder");
> props.put("partitioner.class", "example.kafka.SimplePartitioner");
> props.put("request.required.acks", "1");
> ProducerConfig config = new ProducerConfig(props);
> Producer<String, String> producer = new Producer<String, String>(config);
> Random rnd = new Random();
> for(long event=0;event < events;event++) {
> Long runtime = new Date().getTime();
> String msgKey= "192.168.2." + rnd.nextInt(255);
> String msg = runtime + ",www.exmaple.com," + msgKey;
> KeyedMessage<String, String> data = new KeyedMessage<String,
> String>("page_visits", msgKey, msg);
> producer.send(data);
> System.out.println("message sent");
> }
> producer.close();
> }
>
> }
>
Re: Issues Running Kafka Producer Java example
Posted by Stream Processing <sm...@gmail.com>.
I tried running the example on another topic with single partition and
single replica and same example Java Producer works
On Sat, Nov 8, 2014 at 4:25 PM, Hardik Pandya <sm...@gmail.com>
wrote:
> Hello Champs,
>
> I am trying to run first java producer example.
>
> Upon running this example, producer successfully sends the message, at
> least it looks like it does, there is no java dump
>
> But trying to verify the messages on consumer side - it does not return
> any data sent by producer
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+Producer+Example
>
> here's my Producer code - thanks in advance!
>
> package example.kafka;
>
> import java.util.Date;
> import java.util.Properties;
> import java.util.Random;
>
> import kafka.javaapi.producer.Producer;
> import kafka.producer.KeyedMessage;
> import kafka.producer.ProducerConfig;
>
> public class TestProducer {
>
> /**
> * @param args
> */
> public static void main(String[] args) {
> // TODO Auto-generated method stub
> long events = Long.parseLong(args[0]);
> Properties props = new Properties();
> props.put("metadata.broker.list", "10.0.2.15:9092,10.0.2.15:9093,
> 10.0.2.15:9094,10.0.2.15:9095");
> props.put("serializer.class", "kafka.serializer.StringEncoder");
> props.put("partitioner.class", "example.kafka.SimplePartitioner");
> props.put("request.required.acks", "1");
> ProducerConfig config = new ProducerConfig(props);
> Producer<String, String> producer = new Producer<String, String>(config);
> Random rnd = new Random();
> for(long event=0;event < events;event++) {
> Long runtime = new Date().getTime();
> String msgKey= "192.168.2." + rnd.nextInt(255);
> String msg = runtime + ",www.exmaple.com," + msgKey;
> KeyedMessage<String, String> data = new KeyedMessage<String,
> String>("page_visits", msgKey, msg);
> producer.send(data);
> System.out.println("message sent");
> }
> producer.close();
> }
>
> }
>
>
Re: Issues Running Kafka Producer Java example
Posted by Chia-Chun Shih <ch...@gmail.com>.
Please check your consumer's position
http://kafka.apache.org/documentation.html#basic_ops_consumer_lag
2014-11-16 7:09 GMT+08:00 Stream Processing <sm...@gmail.com>:
> anyone?
>
> On Sat, Nov 8, 2014 at 4:25 PM, Hardik Pandya <sm...@gmail.com>
> wrote:
>
> > Hello Champs,
> >
> > I am trying to run first java producer example.
> >
> > Upon running this example, producer successfully sends the message, at
> > least it looks like it does, there is no java dump
> >
> > But trying to verify the messages on consumer side - it does not return
> > any data sent by producer
> >
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+Producer+Example
> >
> > here's my Producer code - thanks in advance!
> >
> > package example.kafka;
> >
> > import java.util.Date;
> > import java.util.Properties;
> > import java.util.Random;
> >
> > import kafka.javaapi.producer.Producer;
> > import kafka.producer.KeyedMessage;
> > import kafka.producer.ProducerConfig;
> >
> > public class TestProducer {
> >
> > /**
> > * @param args
> > */
> > public static void main(String[] args) {
> > // TODO Auto-generated method stub
> > long events = Long.parseLong(args[0]);
> > Properties props = new Properties();
> > props.put("metadata.broker.list", "10.0.2.15:9092,10.0.2.15:9093,
> > 10.0.2.15:9094,10.0.2.15:9095");
> > props.put("serializer.class", "kafka.serializer.StringEncoder");
> > props.put("partitioner.class", "example.kafka.SimplePartitioner");
> > props.put("request.required.acks", "1");
> > ProducerConfig config = new ProducerConfig(props);
> > Producer<String, String> producer = new Producer<String, String>(config);
> > Random rnd = new Random();
> > for(long event=0;event < events;event++) {
> > Long runtime = new Date().getTime();
> > String msgKey= "192.168.2." + rnd.nextInt(255);
> > String msg = runtime + ",www.exmaple.com," + msgKey;
> > KeyedMessage<String, String> data = new KeyedMessage<String,
> > String>("page_visits", msgKey, msg);
> > producer.send(data);
> > System.out.println("message sent");
> > }
> > producer.close();
> > }
> >
> > }
> >
> >
>
Re: Issues Running Kafka Producer Java example
Posted by Stream Processing <sm...@gmail.com>.
anyone?
On Sat, Nov 8, 2014 at 4:25 PM, Hardik Pandya <sm...@gmail.com>
wrote:
> Hello Champs,
>
> I am trying to run first java producer example.
>
> Upon running this example, producer successfully sends the message, at
> least it looks like it does, there is no java dump
>
> But trying to verify the messages on consumer side - it does not return
> any data sent by producer
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+Producer+Example
>
> here's my Producer code - thanks in advance!
>
> package example.kafka;
>
> import java.util.Date;
> import java.util.Properties;
> import java.util.Random;
>
> import kafka.javaapi.producer.Producer;
> import kafka.producer.KeyedMessage;
> import kafka.producer.ProducerConfig;
>
> public class TestProducer {
>
> /**
> * @param args
> */
> public static void main(String[] args) {
> // TODO Auto-generated method stub
> long events = Long.parseLong(args[0]);
> Properties props = new Properties();
> props.put("metadata.broker.list", "10.0.2.15:9092,10.0.2.15:9093,
> 10.0.2.15:9094,10.0.2.15:9095");
> props.put("serializer.class", "kafka.serializer.StringEncoder");
> props.put("partitioner.class", "example.kafka.SimplePartitioner");
> props.put("request.required.acks", "1");
> ProducerConfig config = new ProducerConfig(props);
> Producer<String, String> producer = new Producer<String, String>(config);
> Random rnd = new Random();
> for(long event=0;event < events;event++) {
> Long runtime = new Date().getTime();
> String msgKey= "192.168.2." + rnd.nextInt(255);
> String msg = runtime + ",www.exmaple.com," + msgKey;
> KeyedMessage<String, String> data = new KeyedMessage<String,
> String>("page_visits", msgKey, msg);
> producer.send(data);
> System.out.println("message sent");
> }
> producer.close();
> }
>
> }
>
>