You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Steve Gao <st...@yahoo.com> on 2009/03/18 00:22:09 UTC
Does HDFS provide a way to append A file to B ?
I need to append file A to file B in HDFS without downloading/uploading them to local disk. Is there a way?
Re: Does HDFS provide a way to append A file to B ?
Posted by Bryan Duxbury <br...@rapleaf.com>.
No. There isn't *any* version of Hadoop with a (stable) append command.
On Mar 17, 2009, at 5:08 PM, Steve Gao wrote:
> Thanks, Bryan. Does 0.18.3 has built-in "append" command?
>
> --- On Tue, 3/17/09, Bryan Duxbury <br...@rapleaf.com> wrote:
> From: Bryan Duxbury <br...@rapleaf.com>
> Subject: Re: Does HDFS provide a way to append A file to B ?
> To: core-user@hadoop.apache.org
> Date: Tuesday, March 17, 2009, 8:04 PM
>
> I believe the last word on appends right now is that the patch that
> was
> committed broke a lot of other things, so it's been disabled. As
> such, there
> is no working append in HDFS, and certainly not in hadoop-17.x.
>
> -Bryan
>
> On Mar 17, 2009, at 4:50 PM, Steve Gao wrote:
>
>> Thanks, but I was told there is an append command, isn't there? But I
> don't know how to apply this patch
> https://issues.apache.org/jira/browse/HADOOP-1700
>>
>> --- On Tue, 3/17/09, Bo Shi <bs...@visiblemeasures.com> wrote:
>> From: Bo Shi <bs...@visiblemeasures.com>
>> Subject: Re: Does HDFS provide a way to append A file to B ?
>> To: core-user@hadoop.apache.org
>> Date: Tuesday, March 17, 2009, 7:42 PM
>>
>> what about an identity mapper taking A and B as inputs? this will
>> likely mix rows of A and B together though...
>>
>> On Tue, Mar 17, 2009 at 7:35 PM, Steve Gao <st...@yahoo.com>
> wrote:
>>> BTW, I am using hadoop 0.17.0 and jdk 1.6
>>>
>>> --- On Tue, 3/17/09, Steve Gao <st...@yahoo.com> wrote:
>>> From: Steve Gao <st...@yahoo.com>
>>> Subject: Does HDFS provide a way to append A file to B ?
>>> To: core-user@hadoop.apache.org
>>> Date: Tuesday, March 17, 2009, 7:22 PM
>>>
>>> I need to append file A to file B in HDFS without
> downloading/uploading
>> them to
>>> local disk. Is there a way?
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>
>
Re: Does HDFS provide a way to append A file to B ?
Posted by Steve Gao <st...@yahoo.com>.
Thanks, Bryan. Does 0.18.3 has built-in "append" command?
--- On Tue, 3/17/09, Bryan Duxbury <br...@rapleaf.com> wrote:
From: Bryan Duxbury <br...@rapleaf.com>
Subject: Re: Does HDFS provide a way to append A file to B ?
To: core-user@hadoop.apache.org
Date: Tuesday, March 17, 2009, 8:04 PM
I believe the last word on appends right now is that the patch that was
committed broke a lot of other things, so it's been disabled. As such, there
is no working append in HDFS, and certainly not in hadoop-17.x.
-Bryan
On Mar 17, 2009, at 4:50 PM, Steve Gao wrote:
> Thanks, but I was told there is an append command, isn't there? But I
don't know how to apply this patch
https://issues.apache.org/jira/browse/HADOOP-1700
>
> --- On Tue, 3/17/09, Bo Shi <bs...@visiblemeasures.com> wrote:
> From: Bo Shi <bs...@visiblemeasures.com>
> Subject: Re: Does HDFS provide a way to append A file to B ?
> To: core-user@hadoop.apache.org
> Date: Tuesday, March 17, 2009, 7:42 PM
>
> what about an identity mapper taking A and B as inputs? this will
> likely mix rows of A and B together though...
>
> On Tue, Mar 17, 2009 at 7:35 PM, Steve Gao <st...@yahoo.com>
wrote:
>> BTW, I am using hadoop 0.17.0 and jdk 1.6
>>
>> --- On Tue, 3/17/09, Steve Gao <st...@yahoo.com> wrote:
>> From: Steve Gao <st...@yahoo.com>
>> Subject: Does HDFS provide a way to append A file to B ?
>> To: core-user@hadoop.apache.org
>> Date: Tuesday, March 17, 2009, 7:22 PM
>>
>> I need to append file A to file B in HDFS without
downloading/uploading
> them to
>> local disk. Is there a way?
>>
>>
>>
>>
>>
>>
>>
>
>
>
Re: Does HDFS provide a way to append A file to B ?
Posted by Bryan Duxbury <br...@rapleaf.com>.
I believe the last word on appends right now is that the patch that
was committed broke a lot of other things, so it's been disabled. As
such, there is no working append in HDFS, and certainly not in
hadoop-17.x.
-Bryan
On Mar 17, 2009, at 4:50 PM, Steve Gao wrote:
> Thanks, but I was told there is an append command, isn't there? But
> I don't know how to apply this patch https://issues.apache.org/jira/
> browse/HADOOP-1700
>
> --- On Tue, 3/17/09, Bo Shi <bs...@visiblemeasures.com> wrote:
> From: Bo Shi <bs...@visiblemeasures.com>
> Subject: Re: Does HDFS provide a way to append A file to B ?
> To: core-user@hadoop.apache.org
> Date: Tuesday, March 17, 2009, 7:42 PM
>
> what about an identity mapper taking A and B as inputs? this will
> likely mix rows of A and B together though...
>
> On Tue, Mar 17, 2009 at 7:35 PM, Steve Gao <st...@yahoo.com>
> wrote:
>> BTW, I am using hadoop 0.17.0 and jdk 1.6
>>
>> --- On Tue, 3/17/09, Steve Gao <st...@yahoo.com> wrote:
>> From: Steve Gao <st...@yahoo.com>
>> Subject: Does HDFS provide a way to append A file to B ?
>> To: core-user@hadoop.apache.org
>> Date: Tuesday, March 17, 2009, 7:22 PM
>>
>> I need to append file A to file B in HDFS without downloading/
>> uploading
> them to
>> local disk. Is there a way?
>>
>>
>>
>>
>>
>>
>>
>
>
>
Re: Does HDFS provide a way to append A file to B ?
Posted by Steve Gao <st...@yahoo.com>.
Thanks, but I was told there is an append command, isn't there? But I don't know how to apply this patch https://issues.apache.org/jira/browse/HADOOP-1700
--- On Tue, 3/17/09, Bo Shi <bs...@visiblemeasures.com> wrote:
From: Bo Shi <bs...@visiblemeasures.com>
Subject: Re: Does HDFS provide a way to append A file to B ?
To: core-user@hadoop.apache.org
Date: Tuesday, March 17, 2009, 7:42 PM
what about an identity mapper taking A and B as inputs? this will
likely mix rows of A and B together though...
On Tue, Mar 17, 2009 at 7:35 PM, Steve Gao <st...@yahoo.com> wrote:
> BTW, I am using hadoop 0.17.0 and jdk 1.6
>
> --- On Tue, 3/17/09, Steve Gao <st...@yahoo.com> wrote:
> From: Steve Gao <st...@yahoo.com>
> Subject: Does HDFS provide a way to append A file to B ?
> To: core-user@hadoop.apache.org
> Date: Tuesday, March 17, 2009, 7:22 PM
>
> I need to append file A to file B in HDFS without downloading/uploading
them to
> local disk. Is there a way?
>
>
>
>
>
>
>
Re: Does HDFS provide a way to append A file to B ?
Posted by Bo Shi <bs...@visiblemeasures.com>.
what about an identity mapper taking A and B as inputs? this will
likely mix rows of A and B together though...
On Tue, Mar 17, 2009 at 7:35 PM, Steve Gao <st...@yahoo.com> wrote:
> BTW, I am using hadoop 0.17.0 and jdk 1.6
>
> --- On Tue, 3/17/09, Steve Gao <st...@yahoo.com> wrote:
> From: Steve Gao <st...@yahoo.com>
> Subject: Does HDFS provide a way to append A file to B ?
> To: core-user@hadoop.apache.org
> Date: Tuesday, March 17, 2009, 7:22 PM
>
> I need to append file A to file B in HDFS without downloading/uploading them to
> local disk. Is there a way?
>
>
>
>
>
>
>
Re: Does HDFS provide a way to append A file to B ?
Posted by Steve Gao <st...@yahoo.com>.
BTW, I am using hadoop 0.17.0 and jdk 1.6
--- On Tue, 3/17/09, Steve Gao <st...@yahoo.com> wrote:
From: Steve Gao <st...@yahoo.com>
Subject: Does HDFS provide a way to append A file to B ?
To: core-user@hadoop.apache.org
Date: Tuesday, March 17, 2009, 7:22 PM
I need to append file A to file B in HDFS without downloading/uploading them to
local disk. Is there a way?