[GitHub] [hudi] Athayde3586 commented on pull request #5941: [HUDI-4290] Fix fetchLatestBaseFiles to filter replaced filegroups

[GitBox <gi...@apache.org>]

Athayde3586 commented on PR #5941:
URL: https://github.com/apache/hudi/pull/5941#issuecomment-1164293438

   Aver quien eres
   
   On Thu, Jun 23, 2022, 6:15 AM Sagar Sumit ***@***.***> wrote:
   
   > ***@***.**** commented on this pull request.
   > ------------------------------
   >
   > In
   > hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java
   > <https://github.com/apache/hudi/pull/5941#discussion_r904895593>:
   >
   > > @@ -973,6 +974,7 @@ Stream<FileSlice> fetchAllFileSlices(String partitionPath) {
   >     */
   >    public Stream<HoodieBaseFile> fetchLatestBaseFiles(final String partitionPath) {
   >      return fetchAllStoredFileGroups(partitionPath)
   > +        .filter(fg -> !isFileGroupReplaced(fg))
   >
   > This is needed when metadata is not enabled in presto-hive connector then
   > we let presto do the listing and just fetch base files using this API after
   > adding to the view. getLatestBaseFiles(String partitionStr) does
   > additional work, which we did not want for perf reasons.
   >
   > should we audit all similar APIs regarding compaction and clustering?
   > I did check all other public APIs in this view class. This is the only one.
   >
   > —
   > Reply to this email directly, view it on GitHub
   > <https://github.com/apache/hudi/pull/5941#discussion_r904895593>, or
   > unsubscribe
   > <https://github.com/notifications/unsubscribe-auth/AYYXGWVZVHD7JYD6B5LBPV3VQRBORANCNFSM5ZQBWNIA>
   > .
   > You are receiving this because you are subscribed to this thread.Message
   > ID: ***@***.***>
   >
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org