You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Vinoth Chandar (Jira)" <ji...@apache.org> on 2021/11/24 22:35:00 UTC

[jira] [Commented] (HUDI-2816) Unify file listing method of Spark/Flink/Hive

    [ https://issues.apache.org/jira/browse/HUDI-2816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17448833#comment-17448833 ] 

Vinoth Chandar commented on HUDI-2816:
--------------------------------------

IMO we should fall everything back to `HoodieMetadata` layer to fetch listings. All of the writer code is already doing this!

> Unify file listing method of Spark/Flink/Hive
> ---------------------------------------------
>
>                 Key: HUDI-2816
>                 URL: https://issues.apache.org/jira/browse/HUDI-2816
>             Project: Apache Hudi
>          Issue Type: Improvement
>            Reporter: Gary Li
>            Priority: Major
>              Labels: help-wanted, sev:high
>
> Currently, different engines are using different file listing methods, which might lead to inconsistent behavior. We need to unify these to deliver consistent behavior across different engines.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)