[GitHub] [hudi] zhangyue19921010 commented on pull request #5416: [HUDI-3963] [RFC-51] Improve Hoodie Writing Efficiency Using Lock-Free Message Queue

[GitBox <gi...@apache.org>]

zhangyue19921010 commented on PR #5416:
URL: https://github.com/apache/hudi/pull/5416#issuecomment-1107850315

   This Patch is tested on dev env, I think it's ready for review.
   Although need more java docs here.
   
   Here is the benchmark result between old BoundInMemoryExecutor and new DisruptorExecutor getting **2.0X** performance improved :)
   ```
   Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
   COW Ingestion:                            Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
   ------------------------------------------------------------------------------------------------------------------------
   BoundInMemory Executor                             5557           5607          70          0.2        5556.9       1.0X
   Disruptor Executor                                 2758           2778          28          0.4        2757.7       2.0X
   ```
   
   Please read hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/execution/benchmark/BoundInMemoryExecutorBenchmark.scala for more details.
    
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org