You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Siva Gudavalli <si...@synechron.com> on 2012/10/31 05:58:13 UTC
updates in hdfs
Hi,
Is there any mechanism on how to do updates to a files on hdfs?
Regards
shiv
Re: updates in hdfs
Posted by Harsh J <ha...@cloudera.com>.
Hi,
It is not supported on the 1.x maintenance line where the focus of the
initial implementation work was to get syncs working for HBase alone
(which work perfectly well), and the append functionality is known to
have some rough edges, but the 2.x line has it - in a rewritten form,
and is supported last I checked.
On Wed, Oct 31, 2012 at 9:39 PM, M. C. Srivas <mc...@gmail.com> wrote:
> I was under the impression that file-append was deprecated in HDFS.
>
>
> On Tue, Oct 30, 2012 at 10:13 PM, Harsh J <ha...@cloudera.com> wrote:
>>
>> Shiv,
>>
>> HDFS does have file-append support (i.e. add data at end of a
>> previously closed file). HDFS does not have random-write support (i.e.
>> writing data at a selected offset in the file).
>>
>> What is your use-case though, that you seek to do updates over files
>> on HDFS? Perhaps you are instead looking for HBase, which has a
>> slightly different way of storing things but provides updates+versions
>> in a way over HDFS.
>>
>> On Wed, Oct 31, 2012 at 10:28 AM, Siva Gudavalli
>> <si...@synechron.com> wrote:
>> > Hi,
>> >
>> > Is there any mechanism on how to do updates to a files on hdfs?
>> >
>> > Regards
>> > shiv
>>
>>
>>
>> --
>> Harsh J
>
>
--
Harsh J
Re: updates in hdfs
Posted by Harsh J <ha...@cloudera.com>.
Hi,
It is not supported on the 1.x maintenance line where the focus of the
initial implementation work was to get syncs working for HBase alone
(which work perfectly well), and the append functionality is known to
have some rough edges, but the 2.x line has it - in a rewritten form,
and is supported last I checked.
On Wed, Oct 31, 2012 at 9:39 PM, M. C. Srivas <mc...@gmail.com> wrote:
> I was under the impression that file-append was deprecated in HDFS.
>
>
> On Tue, Oct 30, 2012 at 10:13 PM, Harsh J <ha...@cloudera.com> wrote:
>>
>> Shiv,
>>
>> HDFS does have file-append support (i.e. add data at end of a
>> previously closed file). HDFS does not have random-write support (i.e.
>> writing data at a selected offset in the file).
>>
>> What is your use-case though, that you seek to do updates over files
>> on HDFS? Perhaps you are instead looking for HBase, which has a
>> slightly different way of storing things but provides updates+versions
>> in a way over HDFS.
>>
>> On Wed, Oct 31, 2012 at 10:28 AM, Siva Gudavalli
>> <si...@synechron.com> wrote:
>> > Hi,
>> >
>> > Is there any mechanism on how to do updates to a files on hdfs?
>> >
>> > Regards
>> > shiv
>>
>>
>>
>> --
>> Harsh J
>
>
--
Harsh J
Re: updates in hdfs
Posted by Harsh J <ha...@cloudera.com>.
Hi,
It is not supported on the 1.x maintenance line where the focus of the
initial implementation work was to get syncs working for HBase alone
(which work perfectly well), and the append functionality is known to
have some rough edges, but the 2.x line has it - in a rewritten form,
and is supported last I checked.
On Wed, Oct 31, 2012 at 9:39 PM, M. C. Srivas <mc...@gmail.com> wrote:
> I was under the impression that file-append was deprecated in HDFS.
>
>
> On Tue, Oct 30, 2012 at 10:13 PM, Harsh J <ha...@cloudera.com> wrote:
>>
>> Shiv,
>>
>> HDFS does have file-append support (i.e. add data at end of a
>> previously closed file). HDFS does not have random-write support (i.e.
>> writing data at a selected offset in the file).
>>
>> What is your use-case though, that you seek to do updates over files
>> on HDFS? Perhaps you are instead looking for HBase, which has a
>> slightly different way of storing things but provides updates+versions
>> in a way over HDFS.
>>
>> On Wed, Oct 31, 2012 at 10:28 AM, Siva Gudavalli
>> <si...@synechron.com> wrote:
>> > Hi,
>> >
>> > Is there any mechanism on how to do updates to a files on hdfs?
>> >
>> > Regards
>> > shiv
>>
>>
>>
>> --
>> Harsh J
>
>
--
Harsh J
Re: updates in hdfs
Posted by Harsh J <ha...@cloudera.com>.
Hi,
It is not supported on the 1.x maintenance line where the focus of the
initial implementation work was to get syncs working for HBase alone
(which work perfectly well), and the append functionality is known to
have some rough edges, but the 2.x line has it - in a rewritten form,
and is supported last I checked.
On Wed, Oct 31, 2012 at 9:39 PM, M. C. Srivas <mc...@gmail.com> wrote:
> I was under the impression that file-append was deprecated in HDFS.
>
>
> On Tue, Oct 30, 2012 at 10:13 PM, Harsh J <ha...@cloudera.com> wrote:
>>
>> Shiv,
>>
>> HDFS does have file-append support (i.e. add data at end of a
>> previously closed file). HDFS does not have random-write support (i.e.
>> writing data at a selected offset in the file).
>>
>> What is your use-case though, that you seek to do updates over files
>> on HDFS? Perhaps you are instead looking for HBase, which has a
>> slightly different way of storing things but provides updates+versions
>> in a way over HDFS.
>>
>> On Wed, Oct 31, 2012 at 10:28 AM, Siva Gudavalli
>> <si...@synechron.com> wrote:
>> > Hi,
>> >
>> > Is there any mechanism on how to do updates to a files on hdfs?
>> >
>> > Regards
>> > shiv
>>
>>
>>
>> --
>> Harsh J
>
>
--
Harsh J
Re: updates in hdfs
Posted by "M. C. Srivas" <mc...@gmail.com>.
I was under the impression that file-append was deprecated in HDFS.
On Tue, Oct 30, 2012 at 10:13 PM, Harsh J <ha...@cloudera.com> wrote:
> Shiv,
>
> HDFS does have file-append support (i.e. add data at end of a
> previously closed file). HDFS does not have random-write support (i.e.
> writing data at a selected offset in the file).
>
> What is your use-case though, that you seek to do updates over files
> on HDFS? Perhaps you are instead looking for HBase, which has a
> slightly different way of storing things but provides updates+versions
> in a way over HDFS.
>
> On Wed, Oct 31, 2012 at 10:28 AM, Siva Gudavalli
> <si...@synechron.com> wrote:
> > Hi,
> >
> > Is there any mechanism on how to do updates to a files on hdfs?
> >
> > Regards
> > shiv
>
>
>
> --
> Harsh J
>
Re: updates in hdfs
Posted by "M. C. Srivas" <mc...@gmail.com>.
I was under the impression that file-append was deprecated in HDFS.
On Tue, Oct 30, 2012 at 10:13 PM, Harsh J <ha...@cloudera.com> wrote:
> Shiv,
>
> HDFS does have file-append support (i.e. add data at end of a
> previously closed file). HDFS does not have random-write support (i.e.
> writing data at a selected offset in the file).
>
> What is your use-case though, that you seek to do updates over files
> on HDFS? Perhaps you are instead looking for HBase, which has a
> slightly different way of storing things but provides updates+versions
> in a way over HDFS.
>
> On Wed, Oct 31, 2012 at 10:28 AM, Siva Gudavalli
> <si...@synechron.com> wrote:
> > Hi,
> >
> > Is there any mechanism on how to do updates to a files on hdfs?
> >
> > Regards
> > shiv
>
>
>
> --
> Harsh J
>
Re: updates in hdfs
Posted by "M. C. Srivas" <mc...@gmail.com>.
I was under the impression that file-append was deprecated in HDFS.
On Tue, Oct 30, 2012 at 10:13 PM, Harsh J <ha...@cloudera.com> wrote:
> Shiv,
>
> HDFS does have file-append support (i.e. add data at end of a
> previously closed file). HDFS does not have random-write support (i.e.
> writing data at a selected offset in the file).
>
> What is your use-case though, that you seek to do updates over files
> on HDFS? Perhaps you are instead looking for HBase, which has a
> slightly different way of storing things but provides updates+versions
> in a way over HDFS.
>
> On Wed, Oct 31, 2012 at 10:28 AM, Siva Gudavalli
> <si...@synechron.com> wrote:
> > Hi,
> >
> > Is there any mechanism on how to do updates to a files on hdfs?
> >
> > Regards
> > shiv
>
>
>
> --
> Harsh J
>
Re: updates in hdfs
Posted by "M. C. Srivas" <mc...@gmail.com>.
I was under the impression that file-append was deprecated in HDFS.
On Tue, Oct 30, 2012 at 10:13 PM, Harsh J <ha...@cloudera.com> wrote:
> Shiv,
>
> HDFS does have file-append support (i.e. add data at end of a
> previously closed file). HDFS does not have random-write support (i.e.
> writing data at a selected offset in the file).
>
> What is your use-case though, that you seek to do updates over files
> on HDFS? Perhaps you are instead looking for HBase, which has a
> slightly different way of storing things but provides updates+versions
> in a way over HDFS.
>
> On Wed, Oct 31, 2012 at 10:28 AM, Siva Gudavalli
> <si...@synechron.com> wrote:
> > Hi,
> >
> > Is there any mechanism on how to do updates to a files on hdfs?
> >
> > Regards
> > shiv
>
>
>
> --
> Harsh J
>
Re: updates in hdfs
Posted by Harsh J <ha...@cloudera.com>.
Shiv,
HDFS does have file-append support (i.e. add data at end of a
previously closed file). HDFS does not have random-write support (i.e.
writing data at a selected offset in the file).
What is your use-case though, that you seek to do updates over files
on HDFS? Perhaps you are instead looking for HBase, which has a
slightly different way of storing things but provides updates+versions
in a way over HDFS.
On Wed, Oct 31, 2012 at 10:28 AM, Siva Gudavalli
<si...@synechron.com> wrote:
> Hi,
>
> Is there any mechanism on how to do updates to a files on hdfs?
>
> Regards
> shiv
--
Harsh J
Re: updates in hdfs
Posted by Harsh J <ha...@cloudera.com>.
Shiv,
HDFS does have file-append support (i.e. add data at end of a
previously closed file). HDFS does not have random-write support (i.e.
writing data at a selected offset in the file).
What is your use-case though, that you seek to do updates over files
on HDFS? Perhaps you are instead looking for HBase, which has a
slightly different way of storing things but provides updates+versions
in a way over HDFS.
On Wed, Oct 31, 2012 at 10:28 AM, Siva Gudavalli
<si...@synechron.com> wrote:
> Hi,
>
> Is there any mechanism on how to do updates to a files on hdfs?
>
> Regards
> shiv
--
Harsh J
Re: updates in hdfs
Posted by Harsh J <ha...@cloudera.com>.
Shiv,
HDFS does have file-append support (i.e. add data at end of a
previously closed file). HDFS does not have random-write support (i.e.
writing data at a selected offset in the file).
What is your use-case though, that you seek to do updates over files
on HDFS? Perhaps you are instead looking for HBase, which has a
slightly different way of storing things but provides updates+versions
in a way over HDFS.
On Wed, Oct 31, 2012 at 10:28 AM, Siva Gudavalli
<si...@synechron.com> wrote:
> Hi,
>
> Is there any mechanism on how to do updates to a files on hdfs?
>
> Regards
> shiv
--
Harsh J
Re: updates in hdfs
Posted by Harsh J <ha...@cloudera.com>.
Shiv,
HDFS does have file-append support (i.e. add data at end of a
previously closed file). HDFS does not have random-write support (i.e.
writing data at a selected offset in the file).
What is your use-case though, that you seek to do updates over files
on HDFS? Perhaps you are instead looking for HBase, which has a
slightly different way of storing things but provides updates+versions
in a way over HDFS.
On Wed, Oct 31, 2012 at 10:28 AM, Siva Gudavalli
<si...@synechron.com> wrote:
> Hi,
>
> Is there any mechanism on how to do updates to a files on hdfs?
>
> Regards
> shiv
--
Harsh J