You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Vu Ho (Jira)" <ji...@apache.org> on 2020/10/10 09:17:00 UTC

[jira] [Created] (HUDI-1330) handle prefix filtering at directory level

Vu Ho created HUDI-1330:
---------------------------

             Summary: handle prefix filtering at directory level
                 Key: HUDI-1330
                 URL: https://issues.apache.org/jira/browse/HUDI-1330
             Project: Apache Hudi
          Issue Type: Bug
          Components: DeltaStreamer, Utilities
            Reporter: Vu Ho


The current DFSPathSelector only ignore prefix(_, .) at the file level while files under intermediate directories
E.g. when reading from a Spark structure streaming source which very often consists of a .checkpoint directory, all metadata files should be ignored. This is not the case currently.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)