[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #6516: [HUDI-4729] Fix fq can not be queried in pending compaction when query ro table with spark

[GitBox <gi...@apache.org>]

alexeykudinkin commented on code in PR #6516:
URL: https://github.com/apache/hudi/pull/6516#discussion_r964102374


##########
hudi-common/src/main/java/org/apache/hudi/BaseHoodieTableFileIndex.java:
##########
@@ -297,6 +297,16 @@ private void doRefresh() {
                   .orElse(Collections.emptyList())
               )
           );
+    } else if (tableType.equals(HoodieTableType.MERGE_ON_READ) && queryType.equals(HoodieTableQueryType.READ_OPTIMIZED)) {

Review Comment:
   @microbearz are we on the same page regarding the 2d point that i've made? Quoting it below:
   
   > We should fix it differently though: instead we should fix filterBaseFileAfterPendingCompaction ([REF](https://github.com/apache/hudi/blob/master/hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java#L416)) to filter out file-slice completely if it becomes empty after filtering out the base file. The idea there is that while compaction is running concurrent writer might add new Log files to the FileSlice so that's why we're only filtering the Base file, but if FileSlice has no Log files and it becomes empty after filtering the Base file we should filter it out completely.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org