[jira] [Created] (HADOOP-16673) Add filter parameter to FileSystem>>listFiles

["Attila Magyar (Jira)" <ji...@apache.org>]

Attila Magyar created HADOOP-16673:
--------------------------------------

             Summary: Add filter parameter to FileSystem>>listFiles
                 Key: HADOOP-16673
                 URL: https://issues.apache.org/jira/browse/HADOOP-16673
             Project: Hadoop Common
          Issue Type: Improvement
          Components: fs
            Reporter: Attila Magyar


Currently getting recursively a filtered list of files in a directory is clumsy because filtering should happen afterwards on the result list.

Imagine we want to list all non hidden files recursively.

The non hidden files filter is defined as: 
{code:java}
!name.startsWith("_") && !name.startsWith(".") {code}
 

Then we can do:

 
{code:java}
RemoteIterator<LocatedFileStatus> remoteIterator = fs.listFiles(path, /*recursive*/true);
while (remoteIterator.hasNext()) {
 LocatedFileStatus each = remoteIterator.next();
 if (filter applies to all of the path elements in each) {
   result.add(each);
 }
}
 
{code}
 

For example each of these paths should be skipped:
 * /.a/b/c
 * /a/.b/c
 * /a/b/.c/

It would be lot better to have a filter parameter on listFiles. This is needed to solve HIVE-22411 effectively. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org