You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Susheel Kumar Gadalay <sk...@gmail.com> on 2014/09/23 12:48:08 UTC
FileUtil Copy example
Can somebody give a good example of using Hadoop FileUtil API to copy
set of files from one directory to another directory.
I want to copy only a set of files not all files in the directory and
also I want to use wild character like part_r_*.
Thanks
Susheel Kumar
Re: FileUtil Copy example
Posted by Susheel Kumar Gadalay <sk...@gmail.com>.
Yes, but I am generating multiple output file names with prefix.
So I know their prefix size.
On 9/25/14, João Alves <jo...@5dlab.com> wrote:
> Hey Susheel,
>
> Maybe it would be a good idea to check if the string is long enough for your
> substring operation. (i.e. fstatus[i].getPath().getName().length() >= 10)
>
>
> On 25 Sep 2014, at 07:14, Susheel Kumar Gadalay <sk...@gmail.com>
> wrote:
>
>> I solved it like this. This will move a file from one location to
>> another location.
>>
>> FileStatus[] fstatus = FileSystem.listStatus(new Path(<Old HDFS
>> directory>));
>> for (int i=0; i<fstatus.length; i++) {
>> if (fstatus[i].isFile()) {
>> if (fstatus[i].getPath().getName().substring(0,
>> 10).matches("<some name>"))
>> FileSystem.rename(new Path(<Old HDFS Directory> + "/" +
>> fstatus[i].getPath().getName()), new Path(<New HDFS Directory> + "/" +
>> fstatus[i].getPath().getName()));
>> }
>> }
>>
>> I don't think copy individual file API is available.
>>
>>
>> On 9/23/14, Susheel Kumar Gadalay <sk...@gmail.com> wrote:
>>> Can somebody give a good example of using Hadoop FileUtil API to copy
>>> set of files from one directory to another directory.
>>>
>>> I want to copy only a set of files not all files in the directory and
>>> also I want to use wild character like part_r_*.
>>>
>>> Thanks
>>> Susheel Kumar
>>>
>
>
Re: FileUtil Copy example
Posted by Susheel Kumar Gadalay <sk...@gmail.com>.
Yes, but I am generating multiple output file names with prefix.
So I know their prefix size.
On 9/25/14, João Alves <jo...@5dlab.com> wrote:
> Hey Susheel,
>
> Maybe it would be a good idea to check if the string is long enough for your
> substring operation. (i.e. fstatus[i].getPath().getName().length() >= 10)
>
>
> On 25 Sep 2014, at 07:14, Susheel Kumar Gadalay <sk...@gmail.com>
> wrote:
>
>> I solved it like this. This will move a file from one location to
>> another location.
>>
>> FileStatus[] fstatus = FileSystem.listStatus(new Path(<Old HDFS
>> directory>));
>> for (int i=0; i<fstatus.length; i++) {
>> if (fstatus[i].isFile()) {
>> if (fstatus[i].getPath().getName().substring(0,
>> 10).matches("<some name>"))
>> FileSystem.rename(new Path(<Old HDFS Directory> + "/" +
>> fstatus[i].getPath().getName()), new Path(<New HDFS Directory> + "/" +
>> fstatus[i].getPath().getName()));
>> }
>> }
>>
>> I don't think copy individual file API is available.
>>
>>
>> On 9/23/14, Susheel Kumar Gadalay <sk...@gmail.com> wrote:
>>> Can somebody give a good example of using Hadoop FileUtil API to copy
>>> set of files from one directory to another directory.
>>>
>>> I want to copy only a set of files not all files in the directory and
>>> also I want to use wild character like part_r_*.
>>>
>>> Thanks
>>> Susheel Kumar
>>>
>
>
Re: FileUtil Copy example
Posted by Susheel Kumar Gadalay <sk...@gmail.com>.
Yes, but I am generating multiple output file names with prefix.
So I know their prefix size.
On 9/25/14, João Alves <jo...@5dlab.com> wrote:
> Hey Susheel,
>
> Maybe it would be a good idea to check if the string is long enough for your
> substring operation. (i.e. fstatus[i].getPath().getName().length() >= 10)
>
>
> On 25 Sep 2014, at 07:14, Susheel Kumar Gadalay <sk...@gmail.com>
> wrote:
>
>> I solved it like this. This will move a file from one location to
>> another location.
>>
>> FileStatus[] fstatus = FileSystem.listStatus(new Path(<Old HDFS
>> directory>));
>> for (int i=0; i<fstatus.length; i++) {
>> if (fstatus[i].isFile()) {
>> if (fstatus[i].getPath().getName().substring(0,
>> 10).matches("<some name>"))
>> FileSystem.rename(new Path(<Old HDFS Directory> + "/" +
>> fstatus[i].getPath().getName()), new Path(<New HDFS Directory> + "/" +
>> fstatus[i].getPath().getName()));
>> }
>> }
>>
>> I don't think copy individual file API is available.
>>
>>
>> On 9/23/14, Susheel Kumar Gadalay <sk...@gmail.com> wrote:
>>> Can somebody give a good example of using Hadoop FileUtil API to copy
>>> set of files from one directory to another directory.
>>>
>>> I want to copy only a set of files not all files in the directory and
>>> also I want to use wild character like part_r_*.
>>>
>>> Thanks
>>> Susheel Kumar
>>>
>
>
Re: FileUtil Copy example
Posted by Susheel Kumar Gadalay <sk...@gmail.com>.
Yes, but I am generating multiple output file names with prefix.
So I know their prefix size.
On 9/25/14, João Alves <jo...@5dlab.com> wrote:
> Hey Susheel,
>
> Maybe it would be a good idea to check if the string is long enough for your
> substring operation. (i.e. fstatus[i].getPath().getName().length() >= 10)
>
>
> On 25 Sep 2014, at 07:14, Susheel Kumar Gadalay <sk...@gmail.com>
> wrote:
>
>> I solved it like this. This will move a file from one location to
>> another location.
>>
>> FileStatus[] fstatus = FileSystem.listStatus(new Path(<Old HDFS
>> directory>));
>> for (int i=0; i<fstatus.length; i++) {
>> if (fstatus[i].isFile()) {
>> if (fstatus[i].getPath().getName().substring(0,
>> 10).matches("<some name>"))
>> FileSystem.rename(new Path(<Old HDFS Directory> + "/" +
>> fstatus[i].getPath().getName()), new Path(<New HDFS Directory> + "/" +
>> fstatus[i].getPath().getName()));
>> }
>> }
>>
>> I don't think copy individual file API is available.
>>
>>
>> On 9/23/14, Susheel Kumar Gadalay <sk...@gmail.com> wrote:
>>> Can somebody give a good example of using Hadoop FileUtil API to copy
>>> set of files from one directory to another directory.
>>>
>>> I want to copy only a set of files not all files in the directory and
>>> also I want to use wild character like part_r_*.
>>>
>>> Thanks
>>> Susheel Kumar
>>>
>
>
Re: FileUtil Copy example
Posted by João Alves <jo...@5dlab.com>.
Hey Susheel,
Maybe it would be a good idea to check if the string is long enough for your substring operation. (i.e. fstatus[i].getPath().getName().length() >= 10)
On 25 Sep 2014, at 07:14, Susheel Kumar Gadalay <sk...@gmail.com> wrote:
> I solved it like this. This will move a file from one location to
> another location.
>
> FileStatus[] fstatus = FileSystem.listStatus(new Path(<Old HDFS directory>));
> for (int i=0; i<fstatus.length; i++) {
> if (fstatus[i].isFile()) {
> if (fstatus[i].getPath().getName().substring(0,
> 10).matches("<some name>"))
> FileSystem.rename(new Path(<Old HDFS Directory> + "/" +
> fstatus[i].getPath().getName()), new Path(<New HDFS Directory> + "/" +
> fstatus[i].getPath().getName()));
> }
> }
>
> I don't think copy individual file API is available.
>
>
> On 9/23/14, Susheel Kumar Gadalay <sk...@gmail.com> wrote:
>> Can somebody give a good example of using Hadoop FileUtil API to copy
>> set of files from one directory to another directory.
>>
>> I want to copy only a set of files not all files in the directory and
>> also I want to use wild character like part_r_*.
>>
>> Thanks
>> Susheel Kumar
>>
Re: FileUtil Copy example
Posted by João Alves <jo...@5dlab.com>.
Hey Susheel,
Maybe it would be a good idea to check if the string is long enough for your substring operation. (i.e. fstatus[i].getPath().getName().length() >= 10)
On 25 Sep 2014, at 07:14, Susheel Kumar Gadalay <sk...@gmail.com> wrote:
> I solved it like this. This will move a file from one location to
> another location.
>
> FileStatus[] fstatus = FileSystem.listStatus(new Path(<Old HDFS directory>));
> for (int i=0; i<fstatus.length; i++) {
> if (fstatus[i].isFile()) {
> if (fstatus[i].getPath().getName().substring(0,
> 10).matches("<some name>"))
> FileSystem.rename(new Path(<Old HDFS Directory> + "/" +
> fstatus[i].getPath().getName()), new Path(<New HDFS Directory> + "/" +
> fstatus[i].getPath().getName()));
> }
> }
>
> I don't think copy individual file API is available.
>
>
> On 9/23/14, Susheel Kumar Gadalay <sk...@gmail.com> wrote:
>> Can somebody give a good example of using Hadoop FileUtil API to copy
>> set of files from one directory to another directory.
>>
>> I want to copy only a set of files not all files in the directory and
>> also I want to use wild character like part_r_*.
>>
>> Thanks
>> Susheel Kumar
>>
Re: FileUtil Copy example
Posted by João Alves <jo...@5dlab.com>.
Hey Susheel,
Maybe it would be a good idea to check if the string is long enough for your substring operation. (i.e. fstatus[i].getPath().getName().length() >= 10)
On 25 Sep 2014, at 07:14, Susheel Kumar Gadalay <sk...@gmail.com> wrote:
> I solved it like this. This will move a file from one location to
> another location.
>
> FileStatus[] fstatus = FileSystem.listStatus(new Path(<Old HDFS directory>));
> for (int i=0; i<fstatus.length; i++) {
> if (fstatus[i].isFile()) {
> if (fstatus[i].getPath().getName().substring(0,
> 10).matches("<some name>"))
> FileSystem.rename(new Path(<Old HDFS Directory> + "/" +
> fstatus[i].getPath().getName()), new Path(<New HDFS Directory> + "/" +
> fstatus[i].getPath().getName()));
> }
> }
>
> I don't think copy individual file API is available.
>
>
> On 9/23/14, Susheel Kumar Gadalay <sk...@gmail.com> wrote:
>> Can somebody give a good example of using Hadoop FileUtil API to copy
>> set of files from one directory to another directory.
>>
>> I want to copy only a set of files not all files in the directory and
>> also I want to use wild character like part_r_*.
>>
>> Thanks
>> Susheel Kumar
>>
Re: FileUtil Copy example
Posted by João Alves <jo...@5dlab.com>.
Hey Susheel,
Maybe it would be a good idea to check if the string is long enough for your substring operation. (i.e. fstatus[i].getPath().getName().length() >= 10)
On 25 Sep 2014, at 07:14, Susheel Kumar Gadalay <sk...@gmail.com> wrote:
> I solved it like this. This will move a file from one location to
> another location.
>
> FileStatus[] fstatus = FileSystem.listStatus(new Path(<Old HDFS directory>));
> for (int i=0; i<fstatus.length; i++) {
> if (fstatus[i].isFile()) {
> if (fstatus[i].getPath().getName().substring(0,
> 10).matches("<some name>"))
> FileSystem.rename(new Path(<Old HDFS Directory> + "/" +
> fstatus[i].getPath().getName()), new Path(<New HDFS Directory> + "/" +
> fstatus[i].getPath().getName()));
> }
> }
>
> I don't think copy individual file API is available.
>
>
> On 9/23/14, Susheel Kumar Gadalay <sk...@gmail.com> wrote:
>> Can somebody give a good example of using Hadoop FileUtil API to copy
>> set of files from one directory to another directory.
>>
>> I want to copy only a set of files not all files in the directory and
>> also I want to use wild character like part_r_*.
>>
>> Thanks
>> Susheel Kumar
>>
Re: FileUtil Copy example
Posted by Susheel Kumar Gadalay <sk...@gmail.com>.
I solved it like this. This will move a file from one location to
another location.
FileStatus[] fstatus = FileSystem.listStatus(new Path(<Old HDFS directory>));
for (int i=0; i<fstatus.length; i++) {
if (fstatus[i].isFile()) {
if (fstatus[i].getPath().getName().substring(0,
10).matches("<some name>"))
FileSystem.rename(new Path(<Old HDFS Directory> + "/" +
fstatus[i].getPath().getName()), new Path(<New HDFS Directory> + "/" +
fstatus[i].getPath().getName()));
}
}
I don't think copy individual file API is available.
On 9/23/14, Susheel Kumar Gadalay <sk...@gmail.com> wrote:
> Can somebody give a good example of using Hadoop FileUtil API to copy
> set of files from one directory to another directory.
>
> I want to copy only a set of files not all files in the directory and
> also I want to use wild character like part_r_*.
>
> Thanks
> Susheel Kumar
>
Re: FileUtil Copy example
Posted by Susheel Kumar Gadalay <sk...@gmail.com>.
I solved it like this. This will move a file from one location to
another location.
FileStatus[] fstatus = FileSystem.listStatus(new Path(<Old HDFS directory>));
for (int i=0; i<fstatus.length; i++) {
if (fstatus[i].isFile()) {
if (fstatus[i].getPath().getName().substring(0,
10).matches("<some name>"))
FileSystem.rename(new Path(<Old HDFS Directory> + "/" +
fstatus[i].getPath().getName()), new Path(<New HDFS Directory> + "/" +
fstatus[i].getPath().getName()));
}
}
I don't think copy individual file API is available.
On 9/23/14, Susheel Kumar Gadalay <sk...@gmail.com> wrote:
> Can somebody give a good example of using Hadoop FileUtil API to copy
> set of files from one directory to another directory.
>
> I want to copy only a set of files not all files in the directory and
> also I want to use wild character like part_r_*.
>
> Thanks
> Susheel Kumar
>
Re: FileUtil Copy example
Posted by Susheel Kumar Gadalay <sk...@gmail.com>.
I solved it like this. This will move a file from one location to
another location.
FileStatus[] fstatus = FileSystem.listStatus(new Path(<Old HDFS directory>));
for (int i=0; i<fstatus.length; i++) {
if (fstatus[i].isFile()) {
if (fstatus[i].getPath().getName().substring(0,
10).matches("<some name>"))
FileSystem.rename(new Path(<Old HDFS Directory> + "/" +
fstatus[i].getPath().getName()), new Path(<New HDFS Directory> + "/" +
fstatus[i].getPath().getName()));
}
}
I don't think copy individual file API is available.
On 9/23/14, Susheel Kumar Gadalay <sk...@gmail.com> wrote:
> Can somebody give a good example of using Hadoop FileUtil API to copy
> set of files from one directory to another directory.
>
> I want to copy only a set of files not all files in the directory and
> also I want to use wild character like part_r_*.
>
> Thanks
> Susheel Kumar
>
Re: FileUtil Copy example
Posted by Susheel Kumar Gadalay <sk...@gmail.com>.
I solved it like this. This will move a file from one location to
another location.
FileStatus[] fstatus = FileSystem.listStatus(new Path(<Old HDFS directory>));
for (int i=0; i<fstatus.length; i++) {
if (fstatus[i].isFile()) {
if (fstatus[i].getPath().getName().substring(0,
10).matches("<some name>"))
FileSystem.rename(new Path(<Old HDFS Directory> + "/" +
fstatus[i].getPath().getName()), new Path(<New HDFS Directory> + "/" +
fstatus[i].getPath().getName()));
}
}
I don't think copy individual file API is available.
On 9/23/14, Susheel Kumar Gadalay <sk...@gmail.com> wrote:
> Can somebody give a good example of using Hadoop FileUtil API to copy
> set of files from one directory to another directory.
>
> I want to copy only a set of files not all files in the directory and
> also I want to use wild character like part_r_*.
>
> Thanks
> Susheel Kumar
>