You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by wangxiaowei <wa...@software.ict.ac.cn> on 2008/07/28 13:31:42 UTC
About HDFS` class Path
Dear All,
I need use Hadoop to read all files in a given directory,I wonder
how to know the path is a directory not a file and if it is
how can I get all the files in the directory?
Thanks Very Much.
Re: About HDFS` class Path
Posted by wangxiaowei <wa...@software.ict.ac.cn>.
Thank you very much,it helps me a lot.I will have a try.
----- Original Message -----
From: "Pratyush Banerjee" <pr...@aol.com>
To: <co...@hadoop.apache.org>
Sent: Monday, July 28, 2008 8:41 PM
Subject: Re: About HDFS` class Path
> Hi,
>
> The listPaths API has been depricated for sometime now. So you can use
> the following API's for the purpose.
>
> **org.apache.hadoop.fs.FileSystem.listStatus(org.apache.hadoop.fs.Path f)
>
> the details of the listStatus API is
>
> **public abstract FileStatus <http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/fs/FileStatus.html>[] *listStatus*(Path <http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/fs/Path.html> f)
> throws IOException <http://java.sun.com/j2se/1.5/docs/api/java/io/IOException.html>
>
> List the statuses of the files/directories in the given path if the
> path is a directory.
>
>
> *Parameters:*
> |f| - given path
> *Returns:*
> the statuses of the files/directories in the given patch
> *Throws:*
> |IOException
> <http://java.sun.com/j2se/1.5/docs/api/java/io/IOException.html>|
>
> Also org.apache.hadoop.fs.FileSystem class gives you a method called
> isFile(Path p)
>
> public boolean *isFile*(Path <http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/fs/Path.html> f)
> throws IOException <http://java.sun.com/j2se/1.5/docs/api/java/io/IOException.html>
>
> True iff the named path is a regular file.
>
> *Throws:*
> |IOException
> <http://java.sun.com/j2se/1.5/docs/api/java/io/IOException.html>|
>
> Finally org.apache.hadoop.fs.FileStatus will give you a isDir() function
>
> public boolean *isDir*()
>
> Is this a directory?
>
> *Returns:*
> true if this is a directory
>
> Hope these APIs help you in your purpose.
>
> Pratyush
>
>
>
> wangxiaowei wrote:
>> Thank you ! But could you point out which class the function listPaths belongs to.I can not find it in Path ,FileStatus and FileSystem in hadoop-0.17.1.
>>
>>
>> ----- Original Message -----
>> From: "Andreas Kostyrka" <an...@kostyrka.org>
>> To: <co...@hadoop.apache.org>
>> Sent: Monday, July 28, 2008 7:37 PM
>> Subject: Re: About HDFS` class Path
>>
>>
>
>
Re: About HDFS` class Path
Posted by Pratyush Banerjee <pr...@aol.com>.
Hi,
The listPaths API has been depricated for sometime now. So you can use
the following API's for the purpose.
**org.apache.hadoop.fs.FileSystem.listStatus(org.apache.hadoop.fs.Path f)
the details of the listStatus API is
**public abstract FileStatus <http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/fs/FileStatus.html>[] *listStatus*(Path <http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/fs/Path.html> f)
throws IOException <http://java.sun.com/j2se/1.5/docs/api/java/io/IOException.html>
List the statuses of the files/directories in the given path if the
path is a directory.
*Parameters:*
|f| - given path
*Returns:*
the statuses of the files/directories in the given patch
*Throws:*
|IOException
<http://java.sun.com/j2se/1.5/docs/api/java/io/IOException.html>|
Also org.apache.hadoop.fs.FileSystem class gives you a method called
isFile(Path p)
public boolean *isFile*(Path <http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/fs/Path.html> f)
throws IOException <http://java.sun.com/j2se/1.5/docs/api/java/io/IOException.html>
True iff the named path is a regular file.
*Throws:*
|IOException
<http://java.sun.com/j2se/1.5/docs/api/java/io/IOException.html>|
Finally org.apache.hadoop.fs.FileStatus will give you a isDir() function
public boolean *isDir*()
Is this a directory?
*Returns:*
true if this is a directory
Hope these APIs help you in your purpose.
Pratyush
wangxiaowei wrote:
> Thank you ! But could you point out which class the function listPaths belongs to.I can not find it in Path ,FileStatus and FileSystem in hadoop-0.17.1.
>
>
> ----- Original Message -----
> From: "Andreas Kostyrka" <an...@kostyrka.org>
> To: <co...@hadoop.apache.org>
> Sent: Monday, July 28, 2008 7:37 PM
> Subject: Re: About HDFS` class Path
>
>
Re: About HDFS` class Path
Posted by wangxiaowei <wa...@software.ict.ac.cn>.
Thank you ! But could you point out which class the function listPaths belongs to.I can not find it in Path ,FileStatus and FileSystem in hadoop-0.17.1.
----- Original Message -----
From: "Andreas Kostyrka" <an...@kostyrka.org>
To: <co...@hadoop.apache.org>
Sent: Monday, July 28, 2008 7:37 PM
Subject: Re: About HDFS` class Path
Re: About HDFS` class Path
Posted by Andreas Kostyrka <an...@kostyrka.org>.
On Monday 28 July 2008 13:31:42 wangxiaowei wrote:
> Dear All,
> I need use Hadoop to read all files in a given directory,I wonder
> how to know the path is a directory not a file and if it is
> how can I get all the files in the directory?
> Thanks Very Much.
getFileStatus and listPaths should be enough to get you going.
Andreas