You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/10/08 03:03:34 UTC

[GitHub] [hudi] danny0405 commented on pull request #6384: [HUDI-4613] Avoid the use of regex expressions when call hoodieFileGroup#addLogFile function

danny0405 commented on PR #6384:
URL: https://github.com/apache/hudi/pull/6384#issuecomment-1272208595

   > @guanziyue Thanks for your positive feedback. IIUC, this improvement is effective for both Flink/Spark Streaming jobs when build `FileSystemView`. And the time saved is also considerable as @ThinkerLei mentioned above. Of course, it involve some additional memory cost. I totally agree gatekeeper's concerned especially about Flink engine, the restart cost will not be accepted when OOM. Actually in our prod cluster, we did not observe some extra OOM due to this change. Anyway, I think this is one choice for performance improvement. FYI.
   
   Thanks for the feedback, can we have some numbers about the additional memory overhead here ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org