You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2020/07/15 22:19:25 UTC

[GitHub] [hudi] bvaradar commented on issue #1830: [SUPPORT] Processing time gradually increases while using Spark Streaming

bvaradar commented on issue #1830:
URL: https://github.com/apache/hudi/issues/1830#issuecomment-659044105


   From the highlighted section in your spark UI image, it looks like there is an increase during index lookup. Between 2 runs, there is  an increase of 10 sec (around 4%). Is this the same pattern you were seeing in earlier runs. Also, I am not able to clearly see the difference between subsequent runs w.r.t total processing delay. My question is : Are you seeing more than 10 secs difference in total processing delay between subsequent runs ? If so, can you try 0.5.3 release where we fixed driver side performance issues and we have heard confirmation from users regarding speedup. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org