You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by John Lilley <jo...@redpoint.net> on 2014/01/12 00:52:49 UTC
FileSystem iterative or limited alternative to listStatus()
Is there an HDFS file system method for listing a directory contents iteratively, or at least stopping at some limit? We have an application in which the user can type wildcards, which our app expands. However, during interactive phases we only want the first matching file to check its format. There may be folders with many 1000s of files...
Thanks
John
RE: FileSystem iterative or limited alternative to listStatus()
Posted by John Lilley <jo...@redpoint.net>.
Hmmm, that's not right either, but this looks like the following will work since I can stop iterating at any time.
RemoteIterator<LocatedFileStatus> listFiles(Path f, boolean recursive);
Thanks for getting me looking the right direction, anyway!
john
From: Ted Yu [mailto:yuzhihong@gmail.com]
Sent: Saturday, January 11, 2014 4:58 PM
To: common-user@hadoop.apache.org
Subject: Re: FileSystem iterative or limited alternative to listStatus()
Can you utilize the following API ?
public FileStatus[] listStatus(Path f, PathFilter filter)
Cheers
On Sat, Jan 11, 2014 at 3:52 PM, John Lilley <jo...@redpoint.net>> wrote:
Is there an HDFS file system method for listing a directory contents iteratively, or at least stopping at some limit? We have an application in which the user can type wildcards, which our app expands. However, during interactive phases we only want the first matching file to check its format. There may be folders with many 1000s of files...
Thanks
John
RE: FileSystem iterative or limited alternative to listStatus()
Posted by John Lilley <jo...@redpoint.net>.
Hmmm, that's not right either, but this looks like the following will work since I can stop iterating at any time.
RemoteIterator<LocatedFileStatus> listFiles(Path f, boolean recursive);
Thanks for getting me looking the right direction, anyway!
john
From: Ted Yu [mailto:yuzhihong@gmail.com]
Sent: Saturday, January 11, 2014 4:58 PM
To: common-user@hadoop.apache.org
Subject: Re: FileSystem iterative or limited alternative to listStatus()
Can you utilize the following API ?
public FileStatus[] listStatus(Path f, PathFilter filter)
Cheers
On Sat, Jan 11, 2014 at 3:52 PM, John Lilley <jo...@redpoint.net>> wrote:
Is there an HDFS file system method for listing a directory contents iteratively, or at least stopping at some limit? We have an application in which the user can type wildcards, which our app expands. However, during interactive phases we only want the first matching file to check its format. There may be folders with many 1000s of files...
Thanks
John
RE: FileSystem iterative or limited alternative to listStatus()
Posted by John Lilley <jo...@redpoint.net>.
Hmmm, that's not right either, but this looks like the following will work since I can stop iterating at any time.
RemoteIterator<LocatedFileStatus> listFiles(Path f, boolean recursive);
Thanks for getting me looking the right direction, anyway!
john
From: Ted Yu [mailto:yuzhihong@gmail.com]
Sent: Saturday, January 11, 2014 4:58 PM
To: common-user@hadoop.apache.org
Subject: Re: FileSystem iterative or limited alternative to listStatus()
Can you utilize the following API ?
public FileStatus[] listStatus(Path f, PathFilter filter)
Cheers
On Sat, Jan 11, 2014 at 3:52 PM, John Lilley <jo...@redpoint.net>> wrote:
Is there an HDFS file system method for listing a directory contents iteratively, or at least stopping at some limit? We have an application in which the user can type wildcards, which our app expands. However, during interactive phases we only want the first matching file to check its format. There may be folders with many 1000s of files...
Thanks
John
RE: FileSystem iterative or limited alternative to listStatus()
Posted by John Lilley <jo...@redpoint.net>.
Hmmm, that's not right either, but this looks like the following will work since I can stop iterating at any time.
RemoteIterator<LocatedFileStatus> listFiles(Path f, boolean recursive);
Thanks for getting me looking the right direction, anyway!
john
From: Ted Yu [mailto:yuzhihong@gmail.com]
Sent: Saturday, January 11, 2014 4:58 PM
To: common-user@hadoop.apache.org
Subject: Re: FileSystem iterative or limited alternative to listStatus()
Can you utilize the following API ?
public FileStatus[] listStatus(Path f, PathFilter filter)
Cheers
On Sat, Jan 11, 2014 at 3:52 PM, John Lilley <jo...@redpoint.net>> wrote:
Is there an HDFS file system method for listing a directory contents iteratively, or at least stopping at some limit? We have an application in which the user can type wildcards, which our app expands. However, during interactive phases we only want the first matching file to check its format. There may be folders with many 1000s of files...
Thanks
John
Re: FileSystem iterative or limited alternative to listStatus()
Posted by Ted Yu <yu...@gmail.com>.
Can you utilize the following API ?
public FileStatus[] listStatus(Path f, PathFilter filter)
Cheers
On Sat, Jan 11, 2014 at 3:52 PM, John Lilley <jo...@redpoint.net>wrote:
> Is there an HDFS file system method for listing a directory contents
> iteratively, or at least stopping at some limit? We have an application in
> which the user can type wildcards, which our app expands. However, during
> interactive phases we only want the first matching file to check its
> format. There may be folders with many 1000s of files…
>
> Thanks
>
> John
>
>
>
Re: FileSystem iterative or limited alternative to listStatus()
Posted by Ted Yu <yu...@gmail.com>.
Can you utilize the following API ?
public FileStatus[] listStatus(Path f, PathFilter filter)
Cheers
On Sat, Jan 11, 2014 at 3:52 PM, John Lilley <jo...@redpoint.net>wrote:
> Is there an HDFS file system method for listing a directory contents
> iteratively, or at least stopping at some limit? We have an application in
> which the user can type wildcards, which our app expands. However, during
> interactive phases we only want the first matching file to check its
> format. There may be folders with many 1000s of files…
>
> Thanks
>
> John
>
>
>
Re: FileSystem iterative or limited alternative to listStatus()
Posted by Ted Yu <yu...@gmail.com>.
Can you utilize the following API ?
public FileStatus[] listStatus(Path f, PathFilter filter)
Cheers
On Sat, Jan 11, 2014 at 3:52 PM, John Lilley <jo...@redpoint.net>wrote:
> Is there an HDFS file system method for listing a directory contents
> iteratively, or at least stopping at some limit? We have an application in
> which the user can type wildcards, which our app expands. However, during
> interactive phases we only want the first matching file to check its
> format. There may be folders with many 1000s of files…
>
> Thanks
>
> John
>
>
>
Re: FileSystem iterative or limited alternative to listStatus()
Posted by Ted Yu <yu...@gmail.com>.
Can you utilize the following API ?
public FileStatus[] listStatus(Path f, PathFilter filter)
Cheers
On Sat, Jan 11, 2014 at 3:52 PM, John Lilley <jo...@redpoint.net>wrote:
> Is there an HDFS file system method for listing a directory contents
> iteratively, or at least stopping at some limit? We have an application in
> which the user can type wildcards, which our app expands. However, during
> interactive phases we only want the first matching file to check its
> format. There may be folders with many 1000s of files…
>
> Thanks
>
> John
>
>
>