[GitHub] [hudi] zhangyue19921010 commented on a change in pull request #3719: [HUDI-2489]Tuning HoodieROTablePathFilter by caching hoodieTableFileSystemView, aiming to reduce unnecessary list/get requests

[GitBox <gi...@apache.org>]

zhangyue19921010 commented on a change in pull request #3719:
URL: https://github.com/apache/hudi/pull/3719#discussion_r718205032



##########
File path: hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieROTablePathFilter.java
##########
@@ -175,8 +181,12 @@ public boolean accept(Path path) {
             metaClientCache.put(baseDir.toString(), metaClient);
           }
 
-          fsView = FileSystemViewManager.createInMemoryFileSystemView(engineContext,
-              metaClient, HoodieInputFormatUtils.buildMetadataConfig(getConf()));
+          fsView = hoodieTableFileSystemViewCache.get(baseDir.toString());

Review comment:
       Thanks for your review. 
   As we can see HoodieROTablePathFilter already **create and cache** `HoodieTableMetaClient` at baseDir level, also `setLoadActiveTimelineOnLoad(true)` which will create an active timeline in singleton mode.
   So that IMO no matter we cache the fsView or not, any new created files will not appear in current hoodieROTablePathFilter.
   
   Now  we cached the fsView using above cached meta client and cached active timeline. Maybe can have no bad effect but can reduce unnecessary init action.
   
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org