HDFS Append status ?

[Sangmin Lee <sa...@gmail.com>]

Hi all,

I am wondering if append for hdfs is supported in stable releases.
If not, what is the issue? ( I see code for append is already there in
repository )
Also where can I find the documentation of current semantic of append?
(I see some .doc files from HADOOP-1700 but not sure if they are out-dated)
Can anyone shed some light on this?

Thanks,
Sangmin

Re: HDFS Append status ?

[Dhruba Borthakur <dh...@gmail.com>]

My opinion is that HADOOP-5744 does nor change the major semantics that were
laid out in HADOOP-1700... e.g single appender, new reads see flushed data,
etc.etc If you can post your design on hdfs byzantine fault tolerant and
then describe the assumptions of hdfs-appends that it relies on, that will
be great.

thanks,
dhruba


On Mon, May 4, 2009 at 10:29 AM, Sangmin Lee <sa...@gmail.com> wrote:

> Dhruba,
>
> I'm working on making hdfs byzantine fault tolerant and was wondering if I
> have to account for current implementation of append.
> If all append semantic and design will be changed (as HADOOP-5744
> indicated),
> I might have to wait for it to come out.
> If the change won't be significant, I will probably continue with current
> implementation.
>
> Do you have any comment on this?
> Thanks,
>
> Sangmin
>
> On Mon, May 4, 2009 at 3:20 AM, Dhruba Borthakur <dh...@gmail.com> wrote:
>
> > The code for "append" is not supported in stable releases. The .doc from
> > HADOOP-1700 explains the semantics of "append". There are quite a few
> open
> > issues that need to be addressed before "append" is production ready for
> > general applications, i.e. HADOOP-4663, HADOOP-4379, HADOOP-5027 among
> > others.
> >
> > what type of application do you have that wants to use "append"?
> >
> > thanks
> > dhruba
> >
> >
> > On Mon, May 4, 2009 at 12:55 AM, Sangmin Lee <sa...@gmail.com>
> > wrote:
> >
> > > Hi all,
> > >
> > > I am wondering if append for hdfs is supported in stable releases.
> > > If not, what is the issue? ( I see code for append is already there in
> > > repository )
> > > Also where can I find the documentation of current semantic of append?
> > > (I see some .doc files from HADOOP-1700 but not sure if they are
> > out-dated)
> > > Can anyone shed some light on this?
> > >
> > > Thanks,
> > > Sangmin
> > >
> >
>

Re: HDFS Append status ?

[Sangmin Lee <sa...@gmail.com>]

Dhruba,

I'm working on making hdfs byzantine fault tolerant and was wondering if I
have to account for current implementation of append.
If all append semantic and design will be changed (as HADOOP-5744
indicated),
I might have to wait for it to come out.
If the change won't be significant, I will probably continue with current
implementation.

Do you have any comment on this?
Thanks,

Sangmin

On Mon, May 4, 2009 at 3:20 AM, Dhruba Borthakur <dh...@gmail.com> wrote:

> The code for "append" is not supported in stable releases. The .doc from
> HADOOP-1700 explains the semantics of "append". There are quite a few open
> issues that need to be addressed before "append" is production ready for
> general applications, i.e. HADOOP-4663, HADOOP-4379, HADOOP-5027 among
> others.
>
> what type of application do you have that wants to use "append"?
>
> thanks
> dhruba
>
>
> On Mon, May 4, 2009 at 12:55 AM, Sangmin Lee <sa...@gmail.com>
> wrote:
>
> > Hi all,
> >
> > I am wondering if append for hdfs is supported in stable releases.
> > If not, what is the issue? ( I see code for append is already there in
> > repository )
> > Also where can I find the documentation of current semantic of append?
> > (I see some .doc files from HADOOP-1700 but not sure if they are
> out-dated)
> > Can anyone shed some light on this?
> >
> > Thanks,
> > Sangmin
> >
>

Re: HDFS Append status ?

[Dhruba Borthakur <dh...@gmail.com>]

The code for "append" is not supported in stable releases. The .doc from
HADOOP-1700 explains the semantics of "append". There are quite a few open
issues that need to be addressed before "append" is production ready for
general applications, i.e. HADOOP-4663, HADOOP-4379, HADOOP-5027 among
others.

what type of application do you have that wants to use "append"?

thanks
dhruba


On Mon, May 4, 2009 at 12:55 AM, Sangmin Lee <sa...@gmail.com> wrote:

> Hi all,
>
> I am wondering if append for hdfs is supported in stable releases.
> If not, what is the issue? ( I see code for append is already there in
> repository )
> Also where can I find the documentation of current semantic of append?
> (I see some .doc files from HADOOP-1700 but not sure if they are out-dated)
> Can anyone shed some light on this?
>
> Thanks,
> Sangmin
>