You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Ted <r6...@gmail.com> on 2016/06/17 18:49:30 UTC
what causes "Memst oreFlusherChore requesting flush" messages?
Wondering if anyone knows the cause of messages like :
"MemstoreFlusherChore requesting flush of
<TABLE>,,1465289533949.94286e3c616b8b4635c9f0665e424a05. because
<COLUMN_FAMILY> has an old edit so flush to free WALs after random delay
265357ms"
My log files seem to be filled with them, I think they account for over 50%
of the lines in my log files.
--
Ted.
Re: what causes "Memst oreFlusherChore requesting flush" messages?
Posted by Ted <r6...@gmail.com>.
so what you're saying is that everyone will see at least 1 of those lines
roughly every hour no matter what, and under higher load you'll see it when
ever limits are hit for either WAL or Memstore causing flushes.
So by the sounds of it, it's perfectly normal to be seeing a slew of those
in the log files and they can more or less be safely ignore.
thanks.
On Fri, Jun 17, 2016 at 12:52 PM, Vladimir Rodionov <vl...@gmail.com>
wrote:
> This is done to prevent runaway WAL files. WAL file can't be deleted if
> some unflushed edits from this file exist in RS memstore.
> In case of a high load this may lead to accumulation of a large number of
> WAL files in a file system.
>
> -Vlad
>
> On Fri, Jun 17, 2016 at 12:49 PM, Vladimir Rodionov <
> vladrodionov@gmail.com>
> wrote:
>
> > >> has an old edit
> >
> > Yes, this is what PeriodicMemstoreFlusher does: it checks if there are
> > edits older than (1h, but configurable) in memstore
> > and flushes this memstore if find old edits.
> >
> > -Vlad
> >
> > On Fri, Jun 17, 2016 at 11:49 AM, Ted <r6...@gmail.com> wrote:
> >
> >> Wondering if anyone knows the cause of messages like :
> >>
> >> "MemstoreFlusherChore requesting flush of
> >> <TABLE>,,1465289533949.94286e3c616b8b4635c9f0665e424a05. because
> >> <COLUMN_FAMILY> has an old edit so flush to free WALs after random delay
> >> 265357ms"
> >>
> >> My log files seem to be filled with them, I think they account for over
> >> 50%
> >> of the lines in my log files.
> >>
> >> --
> >> Ted.
> >>
> >
> >
>
--
Ted.
Re: what causes "Memst oreFlusherChore requesting flush" messages?
Posted by Vladimir Rodionov <vl...@gmail.com>.
This is done to prevent runaway WAL files. WAL file can't be deleted if
some unflushed edits from this file exist in RS memstore.
In case of a high load this may lead to accumulation of a large number of
WAL files in a file system.
-Vlad
On Fri, Jun 17, 2016 at 12:49 PM, Vladimir Rodionov <vl...@gmail.com>
wrote:
> >> has an old edit
>
> Yes, this is what PeriodicMemstoreFlusher does: it checks if there are
> edits older than (1h, but configurable) in memstore
> and flushes this memstore if find old edits.
>
> -Vlad
>
> On Fri, Jun 17, 2016 at 11:49 AM, Ted <r6...@gmail.com> wrote:
>
>> Wondering if anyone knows the cause of messages like :
>>
>> "MemstoreFlusherChore requesting flush of
>> <TABLE>,,1465289533949.94286e3c616b8b4635c9f0665e424a05. because
>> <COLUMN_FAMILY> has an old edit so flush to free WALs after random delay
>> 265357ms"
>>
>> My log files seem to be filled with them, I think they account for over
>> 50%
>> of the lines in my log files.
>>
>> --
>> Ted.
>>
>
>
Re: what causes "Memst oreFlusherChore requesting flush" messages?
Posted by Vladimir Rodionov <vl...@gmail.com>.
>> has an old edit
Yes, this is what PeriodicMemstoreFlusher does: it checks if there are
edits older than (1h, but configurable) in memstore
and flushes this memstore if find old edits.
-Vlad
On Fri, Jun 17, 2016 at 11:49 AM, Ted <r6...@gmail.com> wrote:
> Wondering if anyone knows the cause of messages like :
>
> "MemstoreFlusherChore requesting flush of
> <TABLE>,,1465289533949.94286e3c616b8b4635c9f0665e424a05. because
> <COLUMN_FAMILY> has an old edit so flush to free WALs after random delay
> 265357ms"
>
> My log files seem to be filled with them, I think they account for over 50%
> of the lines in my log files.
>
> --
> Ted.
>