[GitHub] [hudi] nsivabalan commented on pull request #7841: [HUDI-5496] Avoid unnecessary file system parsing to initialize metadata table for a new data table

["nsivabalan (via GitHub)" <gi...@apache.org>]

nsivabalan commented on PR #7841:
URL: https://github.com/apache/hudi/pull/7841#issuecomment-1416376130

   @vinothchandar : I have fixed the screen shots w/ red block to show the unnecessary computation w/o the fix. 
   wrt perf improvement, above screenshots are for a small table. But still we do see 4.5 vs 2s (after the fix) for metadata table instantiation + 1st commit. 
   this might be costly for a large table. lets say you have 10000+ files in the first commit.  we avoid listing all files, fetching the commit time from it and filtering it out. 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org