You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by rakesh kothari <rk...@hotmail.com> on 2010/10/21 21:27:13 UTC

Moving files in hdfs using API

Hi,

Is "move" not supported in Hdfs ? I can't find any API for that. Looking at the source code for hadoop CLI it seems like it's implementing move by copying data from src to dest and deleting the src. This could be a time consuming operation.



Thanks,
-Rakesh

Re: Moving files in hdfs using API

Posted by james warren <ja...@rockyou.com>.
Adding hdfs-user@, bcc'ing mapreduce-user@

Try http://hadoop.apache.org/hdfs/docs/current/api/org/apache/hadoop/hdfs/DistributedFileSystem.html#rename(org.apache.hadoop.fs.Path,
org.apache.hadoop.fs.Path)<http://hadoop.apache.org/hdfs/docs/current/api/org/apache/hadoop/hdfs/DistributedFileSystem.html#rename(org.apache.hadoop.fs.Path,+org.apache.hadoop.fs.Path)>
 --
the other rename function is part of an interface implemented by the hadoop
daemons.

(And yes, it acts as a move. =)

cheers,
-James

On Thu, Oct 21, 2010 at 12:45 PM, Shrijeet Paliwal
<sh...@rocketfuel.com>wrote:

> Oh wait a sec, you may be right.
> Rename may not exactly work as move, worth a shot. Excuse me.
>
>
> On Thu, Oct 21, 2010 at 12:41 PM, Shrijeet Paliwal <
> shrijeet@rocketfuel.com> wrote:
>
>> rename : http://hadoop.apache.org/hdfs/docs/current/api/org/apache/hadoop/hdfs/protocol/ClientProtocol.html#rename(java.lang.String,
>> java.lang.String, org.apache.hadoop.fs.Options.Rename...)<http://hadoop.apache.org/hdfs/docs/current/api/org/apache/hadoop/hdfs/protocol/ClientProtocol.html#rename(java.lang.String,+java.lang.String,+org.apache.hadoop.fs.Options.Rename...)>
>>
>>
>> On Thu, Oct 21, 2010 at 12:27 PM, rakesh kothari <
>> rkothari_iit@hotmail.com> wrote:
>>
>>>  Hi,
>>>
>>> Is "move" not supported in Hdfs ? I can't find any API for that. Looking
>>> at the source code for hadoop CLI it seems like it's implementing move by
>>> copying data from src to dest and deleting the src. This could be a time
>>> consuming operation.
>>>
>>>
>>>
>>> Thanks,
>>> -Rakesh
>>>
>>
>>
>

Re: Moving files in hdfs using API

Posted by Shrijeet Paliwal <sh...@rocketfuel.com>.
Oh wait a sec, you may be right.
Rename may not exactly work as move, worth a shot. Excuse me.

On Thu, Oct 21, 2010 at 12:41 PM, Shrijeet Paliwal
<sh...@rocketfuel.com>wrote:

> rename : http://hadoop.apache.org/hdfs/docs/current/api/org/apache/hadoop/hdfs/protocol/ClientProtocol.html#rename(java.lang.String,
> java.lang.String, org.apache.hadoop.fs.Options.Rename...)<http://hadoop.apache.org/hdfs/docs/current/api/org/apache/hadoop/hdfs/protocol/ClientProtocol.html#rename(java.lang.String,+java.lang.String,+org.apache.hadoop.fs.Options.Rename...)>
>
>
> On Thu, Oct 21, 2010 at 12:27 PM, rakesh kothari <rkothari_iit@hotmail.com
> > wrote:
>
>>  Hi,
>>
>> Is "move" not supported in Hdfs ? I can't find any API for that. Looking
>> at the source code for hadoop CLI it seems like it's implementing move by
>> copying data from src to dest and deleting the src. This could be a time
>> consuming operation.
>>
>>
>>
>> Thanks,
>> -Rakesh
>>
>
>

Re: Moving files in hdfs using API

Posted by Shrijeet Paliwal <sh...@rocketfuel.com>.
rename : http://hadoop.apache.org/hdfs/docs/current/api/org/apache/hadoop/hdfs/protocol/ClientProtocol.html#rename(java.lang.String,
java.lang.String, org.apache.hadoop.fs.Options.Rename...)

On Thu, Oct 21, 2010 at 12:27 PM, rakesh kothari
<rk...@hotmail.com>wrote:

>  Hi,
>
> Is "move" not supported in Hdfs ? I can't find any API for that. Looking at
> the source code for hadoop CLI it seems like it's implementing move by
> copying data from src to dest and deleting the src. This could be a time
> consuming operation.
>
>
>
> Thanks,
> -Rakesh
>