You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2021/02/01 10:36:02 UTC
[GitHub] [beam] scwhittle commented on pull request #13782: [BEAM-11657] Reduce reflection calls when reading from Kafka
scwhittle commented on pull request #13782:
URL: https://github.com/apache/beam/pull/13782#issuecomment-770755221
Not sure if there are dashboards but the perf results from console output were:
16:21:21 org.apache.beam.sdk.io.kafka.KafkaIOIT > testKafkaIOReadsAndWritesCorrectlyInStreaming STANDARD_OUT
16:21:21 Load test results for test (ID): 539adecb-21ea-4aaf-935c-95c9bb9b91c5 and timestamp: 2021-01-28T15:02:34.094000000Z:
16:21:21 Metric: Value:
16:21:21 read_time 1.385
16:21:21 write_time 9.316
16:21:21 run_time 10.701
The subsequent run https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/1871/console had results:
20:34:52 org.apache.beam.sdk.io.kafka.KafkaIOIT > testKafkaIOReadsAndWritesCorrectlyInStreaming STANDARD_OUT
20:34:52 Load test results for test (ID): cb47f5d5-0102-49ce-8fd1-73eb3e6bbe40 and timestamp: 2021-01-28T19:16:14.211000000Z:
20:34:52 Metric: Value:
20:34:52 read_time 2.873
20:34:52 write_time 14.97
20:34:52 run_time 17.843
I'm not sure how stable these are, write_time shouldn't directly be effected, but if the pipeline is doing both in parallel the CPU waste on reading could impact write performance as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org