[GitHub] [kafka] mumrah commented on pull request #9008: KAFKA-9629 Use generated protocol for Fetch API

[GitBox <gi...@apache.org>]

mumrah commented on pull request #9008:
URL: https://github.com/apache/kafka/pull/9008#issuecomment-658918847


   Added some basic jmh benchmarks. Here are the preliminary results (run on my laptop, so take with a grain of salt). All these tests are using 1000 topics with 20 partitions each. For FetchResponse, I used static MemoryRecords rather than FileRecords to try and isolate the serialization time.
   
   On `trunk`:
   
   ```
   Benchmark                                        (partitionCount)  (topicCount)  Mode  Cnt        Score        Error  Units
   FetchRequestBenchmark.testConstructFetchRequest                20          1000  avgt   30         3.591 ±      0.046  ns/op
   FetchRequestBenchmark.testSerializeFetchRequest                20          1000  avgt   30  10049872.274 ± 440324.738  ns/op
   FetchResponseBenchmark.testConstructFetchResponse              20          1000  avgt   30         1.911 ±      0.018  ns/op
   FetchResponseBenchmark.testSerializeFetchResponse              20          1000  avgt   30  13693835.230 ± 150935.356  ns/op
   ```
   
   On this branch:
   
   ```
   Benchmark                                        (partitionCount)  (topicCount)  Mode  Cnt        Score        Error  Units
   FetchRequestBenchmark.testConstructFetchRequest                20          1000  avgt   30  4809813.661 ± 190773.702  ns/op
   FetchRequestBenchmark.testSerializeFetchRequest                20          1000  avgt   30  4646758.697 ± 551449.969  ns/op
   FetchResponseBenchmark.testConstructFetchResponse              20          1000  avgt   30  2507813.886 ±  17457.127  ns/op
   FetchResponseBenchmark.testSerializeFetchResponse              20          1000  avgt   30  7231935.691 ± 461221.717  ns/op
   ```
   
   As we expected quite a bit more time is spent during the construction of FetchRequest/FetchResponse due to conversion to existing data structures. We also see a reducing in serialization time since we no longer convert to `Struct` first.
   
   FetchRequest total construction+serialization time is about the same before and after the change, and FetchResponse total time is slightly less after the change.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org