You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by ChingShen <ch...@gmail.com> on 2010/04/02 11:05:53 UTC

More about LogFlusher

Hi,

  Does anyone know more about the
org.apache.hadoop.hbase.regionserver.LogFlusher?
 I don't know why does it just invoke SequenceFile.Writer.sync()? It just
writes a maker into the file.
Can anyone explain it to me please?

Thanks.

Shen

Re: More about LogFlusher

Posted by ChingShen <ch...@gmail.com>.
Thanks, Stack.

  But, I think I want to know what's different between
SequenceFile.Writer.sync() and syncFs() (HDFS-200) ?
 Could someone tell me what does the HLog sync with something?

Shen

On Sat, Apr 3, 2010 at 3:13 AM, Stack <st...@duboce.net> wrote:

> On Fri, Apr 2, 2010 at 10:59 AM, ChingShen <ch...@gmail.com>
> wrote:
> > Thanks, Jean-Daniel
> >
> >   I haven't patched yet, but what intention does it just write a marker
> > into the file?
> >
>
> IIRC, its so that if the file is corrupted, the parse can be picked up
> at other side of the corrupt section as soon as the parse trips over
> the next marker.
> St.Ack
>

Re: More about LogFlusher

Posted by Stack <st...@duboce.net>.
On Fri, Apr 2, 2010 at 10:59 AM, ChingShen <ch...@gmail.com> wrote:
> Thanks, Jean-Daniel
>
>   I haven't patched yet, but what intention does it just write a marker
> into the file?
>

IIRC, its so that if the file is corrupted, the parse can be picked up
at other side of the corrupt section as soon as the parse trips over
the next marker.
St.Ack

Re: More about LogFlusher

Posted by ChingShen <ch...@gmail.com>.
Thanks, Jean-Daniel

   I haven't patched yet, but what intention does it just write a marker
into the file?

Thanks in advance.

Shen


On Sat, Apr 3, 2010 at 1:03 AM, Jean-Daniel Cryans <jd...@apache.org>wrote:

> LogFlusher isn't doing any FS operations, it calls HLog that does them.
>
> HLog calls sync on the SequenceFile.Writer, which is a marker, and
> also calls fsSync if you patched your hadoop (and replaced the hbase's
> hadoop jar with the patched one) with HDFS-200,826,142.
>
> J-D
>
>

Re: More about LogFlusher

Posted by Jean-Daniel Cryans <jd...@apache.org>.
LogFlusher isn't doing any FS operations, it calls HLog that does them.

HLog calls sync on the SequenceFile.Writer, which is a marker, and
also calls fsSync if you patched your hadoop (and replaced the hbase's
hadoop jar with the patched one) with HDFS-200,826,142.

J-D

On Fri, Apr 2, 2010 at 2:05 AM, ChingShen <ch...@gmail.com> wrote:
> Hi,
>
>  Does anyone know more about the
> org.apache.hadoop.hbase.regionserver.LogFlusher?
>  I don't know why does it just invoke SequenceFile.Writer.sync()? It just
> writes a maker into the file.
> Can anyone explain it to me please?
>
> Thanks.
>
> Shen
>