You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/06/22 17:50:42 UTC

[GitHub] [hudi] yihua commented on a diff in pull request #5941: [HUDI-4290] Fix fetchLatestBaseFiles to filter replaced filegroups

yihua commented on code in PR #5941:
URL: https://github.com/apache/hudi/pull/5941#discussion_r904053034


##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java:
##########
@@ -973,6 +974,7 @@ Stream<FileSlice> fetchAllFileSlices(String partitionPath) {
    */
   public Stream<HoodieBaseFile> fetchLatestBaseFiles(final String partitionPath) {
     return fetchAllStoredFileGroups(partitionPath)
+        .filter(fg -> !isFileGroupReplaced(fg))

Review Comment:
   Good catch!  I see that `getLatestBaseFiles(String partitionStr)` filters out the replaced file groups.  Should that API be used in Presto Hive connector?  Also, should we audit all similar APIs regarding compaction and clustering?
   
   Still, to be on par with `fetchLatestBaseFiles()`, this needs to be fixed anyway.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org