[GitHub] pnowojski commented on issue #6697: [hotfix][benchmarks] Add network broadcast benchmark

[GitBox <gi...@apache.org>]

pnowojski commented on issue #6697: [hotfix][benchmarks] Add network broadcast benchmark
URL: https://github.com/apache/flink/pull/6697#issuecomment-421935662
 
 
   Good point with the channel selector. `RecordWriter#broadcastEmit()` is only used for `RecordWriterOutput#emitWatermark` and `RecordWriterOutput#emitStreamStatus`. Those are less frequent, thus not that performance critical. If we want want to cover only one of those, we should cover the usage with `BroadcastPartitioner`. However it might be indeed best to cover both of them?
   
   Regarding adding a  `@Param` to existing benchmarks - I didn't want to do that, because that would add more time overhead when executing benchmarks. Also I do not see a that big value with benchmarking  broadcasting with all of the other parameters combination. The most common `100ms` and `100` channels should be enough. Also I have refactored the tests and `StreamNetworkBroadcastThroughputBenchmarkTest` executes the same tests as `StreamNetworkThroughputBenchmarkTest` (one extends from the other).
   
   I wanted to merge this benchmark before merging your improvement in order to see the speed up in our continuous benchmarking charts. I hope that I didn't duplicate your work :/ 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services