You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-user@hadoop.apache.org by Thamizhannal Paramasivam <th...@gmail.com> on 2012/03/21 11:31:19 UTC

HDFS Path with special char

Hi All,

Can n't we use special character to get paths in HDFS using Hadoop API.

E.g. Path path = new Path("/data/input/20120321*");
I have files  /data/input/20120321000000/a.txt ,
/data/input/20120321010000/b.txt and /data/input/20120321020000/c.txt under
"/data/input/" folder.

I wanted to all the files correponding to a single day. E.g. File:
/data/input date: 20120321

How can I do that using Hadoop API.

Any help would be appreciated.

Thanks,
Thamizh

Re: HDFS Path with special char

Posted by Thamizhannal Paramasivam <th...@gmail.com>.

Thanks a lot Harish. This is what I am looking for.

On Wed, Mar 21, 2012 at 4:46 PM, Harsh J <ha...@cloudera.com> wrote:

> Seems to me like you are looking for globbing your path results out.
> This is possible, just pass your pattern to this API call
> (FileSystem.globStatus instead of FileSystem.listStatus) and then
> iterate over the returned listing:
>
> http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/fs/FileSystem.html#globStatus(org.apache.hadoop.fs.Path)
>
> On Wed, Mar 21, 2012 at 4:01 PM, Thamizhannal Paramasivam
> <th...@gmail.com> wrote:
> > Hi All,
> >
> > Can n't we use special character to get paths in HDFS using Hadoop API.
> >
> > E.g. Path path = new Path("/data/input/20120321*");
> > I have files  /data/input/20120321000000/a.txt ,
> > /data/input/20120321010000/b.txt and /data/input/20120321020000/c.txt
> under
> > "/data/input/" folder.
> >
> > I wanted to all the files correponding to a single day. E.g. File:
> > /data/input date: 20120321
> >
> > How can I do that using Hadoop API.
> >
> > Any help would be appreciated.
> >
> > Thanks,
> > Thamizh
>
>
>
> --
> Harsh J
>

Re: HDFS Path with special char

Posted by Harsh J <ha...@cloudera.com>.

Seems to me like you are looking for globbing your path results out.
This is possible, just pass your pattern to this API call
(FileSystem.globStatus instead of FileSystem.listStatus) and then
iterate over the returned listing:
http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/fs/FileSystem.html#globStatus(org.apache.hadoop.fs.Path)

On Wed, Mar 21, 2012 at 4:01 PM, Thamizhannal Paramasivam
<th...@gmail.com> wrote:
> Hi All,
>
> Can n't we use special character to get paths in HDFS using Hadoop API.
>
> E.g. Path path = new Path("/data/input/20120321*");
> I have files  /data/input/20120321000000/a.txt ,
> /data/input/20120321010000/b.txt and /data/input/20120321020000/c.txt under
> "/data/input/" folder.
>
> I wanted to all the files correponding to a single day. E.g. File:
> /data/input date: 20120321
>
> How can I do that using Hadoop API.
>
> Any help would be appreciated.
>
> Thanks,
> Thamizh

-- 
Harsh J