You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "kaushik srinivas (JIRA)" <ji...@apache.org> on 2018/07/17 10:01:00 UTC
[jira] [Created] (KAFKA-7171) KafkaPerformanceProducer crashes with
same transaction id.
kaushik srinivas created KAFKA-7171:
---------------------------------------
Summary: KafkaPerformanceProducer crashes with same transaction id.
Key: KAFKA-7171
URL: https://issues.apache.org/jira/browse/KAFKA-7171
Project: Kafka
Issue Type: Bug
Affects Versions: 1.0.1
Reporter: kaushik srinivas
Running org.apache.kafka.tools.ProducerPerformance code to performance test the kafka cluster. As a trial cluster has only one broker and zookeeper with 12GB of heap space.
Running 6 producers on 3 machines with same transaction id (2 producers on each node).
Below are the settings of each producer,
kafka-run-class org.apache.kafka.tools.ProducerPerformance --print-metrics --topic perf1 --num-records 9223372036854 --throughput 250000 --record-size 200 --producer-props bootstrap.servers=localhost:9092 buffer.memory=524288000 batch.size=524288
for 2 hours all producers run fine, then suddenly throughput of all producers increase 3 times and 4 producers on 2 nodes crashes with below exceptions,
[2018-07-16 14:00:18,744] ERROR Error executing user-provided callback on message for topic-partition perf1-6: (org.apache.kafka.clients.producer.internals.RecordBatch)
java.lang.ClassCastException: org.apache.kafka.clients.producer.internals.RecordAccumulator$RecordAppendResult cannot be cast to org.apache.kafka.clients.producer.internals.RecordBatch$Thunk
at org.apache.kafka.clients.producer.internals.RecordBatch.done(RecordBatch.java:99)
at org.apache.kafka.clients.producer.internals.Sender.completeBatch(Sender.java:312)
at org.apache.kafka.clients.producer.internals.Sender.handleProduceResponse(Sender.java:272)
at org.apache.kafka.clients.producer.internals.Sender.access$100(Sender.java:57)
at org.apache.kafka.clients.producer.internals.Sender$1.onComplete(Sender.java:358)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:278)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:229)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:134)
at java.lang.Thread.run(Thread.java:748)
First machine (2 producers) run fine.
Need some pointers on this issue.
Queires:
why the throughput is increasing 3 times after 2 hours of duration ?
why the other producers are crashing ?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)