You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Dhruba Borthakur <dh...@yahoo-inc.com> on 2007/06/01 09:01:35 UTC
RE: HDFS, appending-writes status
Hi Ali,
>From what I know, there are quite a few applications that need
append-to-exisiting-file feature. Since there is a demand for it, HDFS will
probably implement it in the near future. I do not know of precise
time-frames yet.
Do you have an application that requires appending-writes to files. If so,
it will be helpful to understand the type and behaviour of this application.
Thanks,
dhruba
-----Original Message-----
From: Ali Salehi [mailto:ali.salehi@epfl.ch]
Sent: Thursday, May 31, 2007 11:46 AM
To: hadoop-user@lucene.apache.org
Subject: HDFS, appending-writes status
Hi,
While I was reading the HDFS docs from the wiki, I bump into this
sentence:
"There is a plan to support appending-writes to a file in future"
I would like to know, if possible, what is the time line for implementing
this future.
Cheers,
AliS
**************************************************************
Ali Salehi, LSIR - Distributed Information Systems Laboratory
EPFL-IC-IIF-LSIR, Bâtiment BC, Station 14, CH-1015 Lausanne, Switzerland.
http://lsirwww.epfl.ch/
email: ali.salehi@epfl.ch
Tel: +41-21-6936656 Fax: +41-21-6938115
Re: HDFS, appending-writes status
Posted by Michael Stack <st...@duboce.net>.
HBase is another application that needs write-append.
Every HBase update is written both to a RAM-based and file-system-based
log. On a period the RAM-based log is flushed to the filesystem. The
RAM-based log and its flushes are used fielding queries.
The sympathetic file-system-based log is meant to act as insurance
against machine crashes. On recovery, the file-system-based log is
supposed to be replayed to catch-up on all updates that happened after
the last RAM-based log flush and before machine crash.
Currently the file-system-based log is being written to HDFS though its
near-to-useless since the log is only seen after its closed.
Thanks,
St.Ack
P.S. Bdbje writing its logs to HDFS would be pretty cool.
Ali Salehi wrote:
> Hi,
> Thanks for your response.
> Actually I want to use it with Berkeley Db java ed. [bdbj].
> BDBJ only uses appends to create its db so I though using hadoop
> would be a viable option considering the scalability requirements
> I'm facing.
>
> Cheers,
> AliS
>
>
>> Hi Ali,
>>
>> >From what I know, there are quite a few applications that need
>> append-to-exisiting-file feature. Since there is a demand for it, HDFS
>> will
>> probably implement it in the near future. I do not know of precise
>> time-frames yet.
>>
>> Do you have an application that requires appending-writes to files. If so,
>> it will be helpful to understand the type and behaviour of this
>> application.
>>
>> Thanks,
>> dhruba
>>
>> -----Original Message-----
>> From: Ali Salehi [mailto:ali.salehi@epfl.ch]
>> Sent: Thursday, May 31, 2007 11:46 AM
>> To: hadoop-user@lucene.apache.org
>> Subject: HDFS, appending-writes status
>>
>> Hi,
>>
>> While I was reading the HDFS docs from the wiki, I bump into this
>> sentence:
>>
>> "There is a plan to support appending-writes to a file in future"
>>
>> I would like to know, if possible, what is the time line for implementing
>> this future.
>>
>>
>> Cheers,
>> AliS
>>
>>
>> **************************************************************
>> Ali Salehi, LSIR - Distributed Information Systems Laboratory
>> EPFL-IC-IIF-LSIR, Bâtiment BC, Station 14, CH-1015 Lausanne, Switzerland.
>> http://lsirwww.epfl.ch/
>> email: ali.salehi@epfl.ch
>> Tel: +41-21-6936656 Fax: +41-21-6938115
>>
>>
>>
>
>
> **************************************************************
> Ali Salehi, LSIR - Distributed Information Systems Laboratory
> EPFL-IC-IIF-LSIR, Bâtiment BC, Station 14, CH-1015 Lausanne, Switzerland.
> http://lsirwww.epfl.ch/
> email: ali.salehi@epfl.ch
> Tel: +41-21-6936656 Fax: +41-21-6938115
>
>
RE: HDFS, appending-writes status
Posted by Ali Salehi <al...@epfl.ch>.
Hi,
Thanks for your response.
Actually I want to use it with Berkeley Db java ed. [bdbj].
BDBJ only uses appends to create its db so I though using hadoop
would be a viable option considering the scalability requirements
I'm facing.
Cheers,
AliS
> Hi Ali,
>
>>>From what I know, there are quite a few applications that need
> append-to-exisiting-file feature. Since there is a demand for it, HDFS
> will
> probably implement it in the near future. I do not know of precise
> time-frames yet.
>
> Do you have an application that requires appending-writes to files. If so,
> it will be helpful to understand the type and behaviour of this
> application.
>
> Thanks,
> dhruba
>
> -----Original Message-----
> From: Ali Salehi [mailto:ali.salehi@epfl.ch]
> Sent: Thursday, May 31, 2007 11:46 AM
> To: hadoop-user@lucene.apache.org
> Subject: HDFS, appending-writes status
>
> Hi,
>
> While I was reading the HDFS docs from the wiki, I bump into this
> sentence:
>
> "There is a plan to support appending-writes to a file in future"
>
> I would like to know, if possible, what is the time line for implementing
> this future.
>
>
> Cheers,
> AliS
>
>
> **************************************************************
> Ali Salehi, LSIR - Distributed Information Systems Laboratory
> EPFL-IC-IIF-LSIR, Bâtiment BC, Station 14, CH-1015 Lausanne, Switzerland.
> http://lsirwww.epfl.ch/
> email: ali.salehi@epfl.ch
> Tel: +41-21-6936656 Fax: +41-21-6938115
>
>
**************************************************************
Ali Salehi, LSIR - Distributed Information Systems Laboratory
EPFL-IC-IIF-LSIR, Bâtiment BC, Station 14, CH-1015 Lausanne, Switzerland.
http://lsirwww.epfl.ch/
email: ali.salehi@epfl.ch
Tel: +41-21-6936656 Fax: +41-21-6938115