You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kafka.apache.org by Dhia Beji <Dh...@keyrus.com> on 2017/11/06 16:52:54 UTC

kafka producer : number of record pushed to topic limited

Hello,

Would you please help me, I m trying to create producer that pushs a large number of json records to topic.

Topic creation :  kafka-topics --create --zookeeper 127.0.0.1:2181  --replication-factor 1 --partitions 100  --topic  testkpa
Topic producer :

            props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "http://<IP>" );
            props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringSerializer");
            props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,"org.apache.kafka.common.serialization.StringSerializer");

            Producer<String, String> producer = new KafkaProducer<String, String>(props)
ProducerRecord<String, String> rec = new ProducerRecord<String, String>(topicName,""+TestStreamCsv.incr ,value);

producer.send(rec)
//

the problem is that the number of records viewed via landoop is limited to 87.

Thank you.


This message has been scanned for malware by Websense. www.websense.com

Re: kafka producer : number of record pushed to topic limited

Posted by Ted Yu <yu...@gmail.com>.

Can you give us some more information ?

Were you using a single node setup (127.0.0.1) ?
Which release of Kafka are you using ?

Anything interesting from broker log ?

On Mon, Nov 6, 2017 at 8:52 AM, Dhia Beji <Dh...@keyrus.com> wrote:

> Hello,
>
> Would you please help me, I m trying to create producer that pushs a large
> number of json records to topic.
>
> Topic creation :  kafka-topics --create --zookeeper 127.0.0.1:2181
> --replication-factor 1 --partitions 100  --topic  testkpa
> Topic producer :
>
>             props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "http://<IP>"
> );
>             props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,
> "org.apache.kafka.common.serialization.StringSerializer");
>             props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,
> "org.apache.kafka.common.serialization.StringSerializer");
>
>             Producer<String, String> producer = new KafkaProducer<String,
> String>(props)
> ProducerRecord<String, String> rec = new ProducerRecord<String,
> String>(topicName,""+TestStreamCsv.incr ,value);
>
> producer.send(rec)
> //
>
> the problem is that the number of records viewed via landoop is limited to
> 87.
>
> Thank you.
>
>
> This message has been scanned for malware by Websense. www.websense.com
>