You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "ThinkerLei (via GitHub)" <gi...@apache.org> on 2023/04/10 05:02:19 UTC
[GitHub] [hudi] ThinkerLei commented on pull request #6384: [HUDI-4613] Avoid the use of regex expressions when call hoodieFileGroup#addLogFile function
ThinkerLei commented on PR #6384:
URL: https://github.com/apache/hudi/pull/6384#issuecomment-1501394656
Now, I did a simple memory overhead test for this change .Some related information, please see the attachment of [HUDI-4613](https://issues.apache.org/jira/browse/HUDI-4613). This change will add about 300~400Byte memory overhead for each logFile. And the FileGroupsCreationTime will decrease from tens of seconds to less than one second ( about 60000 logfiles).
FileGroupsCreationTime with this change:
view.AbstractTableFileSystemView: addFilesToView: NumFiles=59590, NumFileGroups=400, FileGroupsCreationTime=853, StoreTimeTaken=2
FileGroupsCreationTime without this change:
addFilesToView: NumFiles=59961, NumFileGroups=400, FileGroupsCreationTime=11709, StoreTimeTaken=2
@danny0405 @yihua
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org