You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@accumulo.apache.org by Arshak Navruzyan <ar...@gmail.com> on 2014/01/18 00:53:43 UTC
documentation regarding WAL
I am wondering whether the following sections of the docs correctly
describe the current WAL behavior or are these remnants of the time when
the WAL was stored locally.
*2.8. Fault-Tolerance*
The Master will coordinate the copying of write-ahead logs to HDFS so the
logs are available to all tablet servers.
*11.10. Recovery*
Recovery is performed by asking the loggers to copy their write-ahead logs
into HDFS. As the logs are copied, they are also sorted, so that tablets
can easily find their missing updates. The copy/sort status of each file is
displayed on Accumulo monitor status page.
are these accurate or does each tserver directly write it's WAL to HDFS
(vs. having to copy it)?
Re: documentation regarding WAL
Posted by Arshak Navruzyan <ar...@gmail.com>.
I assume we want the docs to reflect the latest(and/or moving forward)
behavior.
I'll make the changes and submit a patch.
On Jan 17, 2014 5:15 PM, "Keith Turner" <ke...@deenlo.com> wrote:
> The documentation is out of date for 1.5. Its ok for 1.4.
>
> In 1.4 tservers write to logger servers which write to local disk.
> Starting in 1.5 tservers write their WALs directly to HDFS.
>
> In 1.4 and1.5 the master will coordinate sorting of logs for recovery
> purposes. In 1.4 loggers do the sorting. In 1.5 tservers do the sorting.
> In both versions the sorted logs are stored in HDFS.
>
>
> On Fri, Jan 17, 2014 at 6:53 PM, Arshak Navruzyan <ar...@gmail.com>wrote:
>
>> I am wondering whether the following sections of the docs correctly
>> describe the current WAL behavior or are these remnants of the time when
>> the WAL was stored locally.
>>
>> *2.8. Fault-Tolerance*
>>
>> The Master will coordinate the copying of write-ahead logs to HDFS so the
>> logs are available to all tablet servers.
>> *11.10. Recovery*
>> Recovery is performed by asking the loggers to copy their write-ahead
>> logs into HDFS. As the logs are copied, they are also sorted, so that
>> tablets can easily find their missing updates. The copy/sort status of each
>> file is displayed on Accumulo monitor status page.
>>
>>
>> are these accurate or does each tserver directly write it's WAL to HDFS
>> (vs. having to copy it)?
>>
>>
>>
>>
>>
>>
>
Re: documentation regarding WAL
Posted by Keith Turner <ke...@deenlo.com>.
The documentation is out of date for 1.5. Its ok for 1.4.
In 1.4 tservers write to logger servers which write to local disk.
Starting in 1.5 tservers write their WALs directly to HDFS.
In 1.4 and1.5 the master will coordinate sorting of logs for recovery
purposes. In 1.4 loggers do the sorting. In 1.5 tservers do the sorting.
In both versions the sorted logs are stored in HDFS.
On Fri, Jan 17, 2014 at 6:53 PM, Arshak Navruzyan <ar...@gmail.com> wrote:
> I am wondering whether the following sections of the docs correctly
> describe the current WAL behavior or are these remnants of the time when
> the WAL was stored locally.
>
> *2.8. Fault-Tolerance*
>
> The Master will coordinate the copying of write-ahead logs to HDFS so the
> logs are available to all tablet servers.
> *11.10. Recovery*
> Recovery is performed by asking the loggers to copy their write-ahead logs
> into HDFS. As the logs are copied, they are also sorted, so that tablets
> can easily find their missing updates. The copy/sort status of each file is
> displayed on Accumulo monitor status page.
>
>
> are these accurate or does each tserver directly write it's WAL to HDFS
> (vs. having to copy it)?
>
>
>
>
>
>