You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Thanh Do <th...@cs.wisc.edu> on 2012/11/01 18:39:49 UTC
SequenceFile syncFs behavior?
Hi all,
Could somebody clarify me the behavior of SequenceFile.syncFs(). From the
source, I saw this function is deprecated, and hsync() or hflush() is
recommended? However, it seems like current stable release of HBase (0.94)
or the cloudera distribution (4.0) uses syncFs for its HLog file.
Would syncFs() guarantee data durability?
I am using hadoop 2.0.
Many thanks,
Thanh Do
Re: SequenceFile syncFs behavior?
Posted by Harsh J <ha...@cloudera.com>.
Hi Thanh Do,
SequenceFile.Writer.syncFs() in 2.x and 2.x-based releases, is
deprecated in favor of hsync() and hflush() new methods but internally
calls hflush itself, so its behavior is the same with regards to
durability as it was before (new metadata entries are created and the
buffer is flushed).
HBase has mechanisms to detect the right method to call (via
reflection) - see the SequenceFileLogWriter#getHFlush(…) method for
example.
On Thu, Nov 1, 2012 at 11:09 PM, Thanh Do <th...@cs.wisc.edu> wrote:
> Hi all,
>
> Could somebody clarify me the behavior of SequenceFile.syncFs(). From the
> source, I saw this function is deprecated, and hsync() or hflush() is
> recommended? However, it seems like current stable release of HBase (0.94)
> or the cloudera distribution (4.0) uses syncFs for its HLog file.
>
> Would syncFs() guarantee data durability?
>
> I am using hadoop 2.0.
>
> Many thanks,
> Thanh Do
>
--
Harsh J
Re: SequenceFile syncFs behavior?
Posted by Harsh J <ha...@cloudera.com>.
Hi Thanh Do,
SequenceFile.Writer.syncFs() in 2.x and 2.x-based releases, is
deprecated in favor of hsync() and hflush() new methods but internally
calls hflush itself, so its behavior is the same with regards to
durability as it was before (new metadata entries are created and the
buffer is flushed).
HBase has mechanisms to detect the right method to call (via
reflection) - see the SequenceFileLogWriter#getHFlush(…) method for
example.
On Thu, Nov 1, 2012 at 11:09 PM, Thanh Do <th...@cs.wisc.edu> wrote:
> Hi all,
>
> Could somebody clarify me the behavior of SequenceFile.syncFs(). From the
> source, I saw this function is deprecated, and hsync() or hflush() is
> recommended? However, it seems like current stable release of HBase (0.94)
> or the cloudera distribution (4.0) uses syncFs for its HLog file.
>
> Would syncFs() guarantee data durability?
>
> I am using hadoop 2.0.
>
> Many thanks,
> Thanh Do
>
--
Harsh J
Re: SequenceFile syncFs behavior?
Posted by Harsh J <ha...@cloudera.com>.
Hi Thanh Do,
SequenceFile.Writer.syncFs() in 2.x and 2.x-based releases, is
deprecated in favor of hsync() and hflush() new methods but internally
calls hflush itself, so its behavior is the same with regards to
durability as it was before (new metadata entries are created and the
buffer is flushed).
HBase has mechanisms to detect the right method to call (via
reflection) - see the SequenceFileLogWriter#getHFlush(…) method for
example.
On Thu, Nov 1, 2012 at 11:09 PM, Thanh Do <th...@cs.wisc.edu> wrote:
> Hi all,
>
> Could somebody clarify me the behavior of SequenceFile.syncFs(). From the
> source, I saw this function is deprecated, and hsync() or hflush() is
> recommended? However, it seems like current stable release of HBase (0.94)
> or the cloudera distribution (4.0) uses syncFs for its HLog file.
>
> Would syncFs() guarantee data durability?
>
> I am using hadoop 2.0.
>
> Many thanks,
> Thanh Do
>
--
Harsh J
Re: SequenceFile syncFs behavior?
Posted by Harsh J <ha...@cloudera.com>.
Hi Thanh Do,
SequenceFile.Writer.syncFs() in 2.x and 2.x-based releases, is
deprecated in favor of hsync() and hflush() new methods but internally
calls hflush itself, so its behavior is the same with regards to
durability as it was before (new metadata entries are created and the
buffer is flushed).
HBase has mechanisms to detect the right method to call (via
reflection) - see the SequenceFileLogWriter#getHFlush(…) method for
example.
On Thu, Nov 1, 2012 at 11:09 PM, Thanh Do <th...@cs.wisc.edu> wrote:
> Hi all,
>
> Could somebody clarify me the behavior of SequenceFile.syncFs(). From the
> source, I saw this function is deprecated, and hsync() or hflush() is
> recommended? However, it seems like current stable release of HBase (0.94)
> or the cloudera distribution (4.0) uses syncFs for its HLog file.
>
> Would syncFs() guarantee data durability?
>
> I am using hadoop 2.0.
>
> Many thanks,
> Thanh Do
>
--
Harsh J