[GitHub] [hudi] kazdy commented on issue #8261: [SUPPORT] How to reduce hoodie commit latency

["kazdy (via GitHub)" <gi...@apache.org>]

kazdy commented on issue #8261:
URL: https://github.com/apache/hudi/issues/8261#issuecomment-1478254161

   in 0.12.1 there was a bug related to hive sync, I also observed increasing processing time, which was mostly spent on reading all files under .hoodie/archived directory
   you can disable hive sync to confirm if this is the issue
   EMR 6.10 comes with 0.12.2 and this is fixed in this version afaik
   
   switching to non global index could also help, depending on the workload
   you can also consider async table services if this is streaming job, but you'd need to rewrite your job and not use forEachBatch() as it only works with inline services


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org