You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2022/11/04 13:11:57 UTC
[GitHub] [kafka] runom opened a new pull request, #12822: KAFKA-14355: Fix integer overflow in ProducerPerformance
runom opened a new pull request, #12822:
URL: https://github.com/apache/kafka/pull/12822
In `kafka-producer-perf-test.sh`, an integer overflow occurs if a fairly large value is passed to `--num-records`. It causes a NegativeArraySizeException error. This PR fixed this problem by changing types.
Before:
```
$ ./bin/kafka-producer-perf-test.sh --topic=test --producer-props bootstrap.servers=localhost:9092 --num-records 2000000000000000 --throughput 1 --record-size 10
Exception in thread "main" java.lang.NegativeArraySizeException
at org.apache.kafka.tools.ProducerPerformance$Stats.<init>(ProducerPerformance.java:354)
at org.apache.kafka.tools.ProducerPerformance.start(ProducerPerformance.java:97)
at org.apache.kafka.tools.ProducerPerformance.main(ProducerPerformance.java:52)
```
After:
```
$ ./bin/kafka-producer-perf-test.sh --topic=test --producer-props bootstrap.servers=localhost:9092 --num-records 2000000000000000 --throughput 1 --record-size 10
7 records sent, 1.3 records/sec (0.00 MB/sec), 123.7 ms avg latency, 684.0 ms max latency.
5 records sent, 1.0 records/sec (0.00 MB/sec), 2.2 ms avg latency, 3.0 ms max latency.
5 records sent, 1.0 records/sec (0.00 MB/sec), 2.2 ms avg latency, 3.0 ms max latency.
5 records sent, 1.0 records/sec (0.00 MB/sec), 2.6 ms avg latency, 3.0 ms max latency.
...
```
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] soarez commented on a diff in pull request #12822: KAFKA-14355: Fix integer overflow in ProducerPerformance
Posted by GitBox <gi...@apache.org>.
soarez commented on code in PR #12822:
URL: https://github.com/apache/kafka/pull/12822#discussion_r1014501624
##########
tools/src/main/java/org/apache/kafka/tools/ProducerPerformance.java:
##########
@@ -350,7 +351,7 @@ public Stats(long numRecords, int reportingInterval) {
this.start = System.currentTimeMillis();
this.windowStart = System.currentTimeMillis();
this.iteration = 0;
- this.sampling = (int) (numRecords / Math.min(numRecords, 500000));
+ this.sampling = numRecords / Math.min(numRecords, 500000);
Review Comment:
We could also limit the value before casting to avoid overflow:
```
this.sampling = (int) Math.min(Integer.MAX_VALUE, numRecords / Math.min(numRecords, 500000));
```
But I think switching `long` as you've elected makes more sense.
I know it's unrelated but just 4 lines below, `this.totalLatency` is being initialized twice, could you remove that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] runom commented on pull request #12822: KAFKA-14355: Fix integer overflow in ProducerPerformance
Posted by GitBox <gi...@apache.org>.
runom commented on PR #12822:
URL: https://github.com/apache/kafka/pull/12822#issuecomment-1304466350
@soarez @showuon Thanks for reviewing! I removed it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon merged pull request #12822: KAFKA-14355: Fix integer overflow in ProducerPerformance
Posted by GitBox <gi...@apache.org>.
showuon merged PR #12822:
URL: https://github.com/apache/kafka/pull/12822
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org