[jira] [Commented] (HBASE-15536) Make AsyncFSWAL as our default WAL

["Duo Zhang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-15536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15538548#comment-15538548 ] 

Duo Zhang commented on HBASE-15536:
-----------------------------------

I filed HBASE-15709 when debugging the problem of {{TestMobStoreScanner}}, where we write a cell with 25MB which exceeded the limit of max packet size(16MB) in DTP. It could be fixed by implementing chunked logic in {{AsyncOneBlockDFSOutputStream}}. But still, if you try a put larger than the block size, the async WAL will fail since it can only handle a one block file.

I think this means the {{AsyncFSWAL}} can not handle all the cases in the current HBase so I'm not sure if it is OK to make it default. And why I leave HBASE-15709 open is that, I do not think it is a good idea to write cells larger than 16MB at one time if we do not enable the mob feature. In our company the limitation is 2MB. And I wonder why mob still writes the whole entry to WAL? For a typical object storage built upon HBase and HDFS, only meta is stored in HBase and the actual data is written to HDFS directly which means it does not need a WAL, it has been persistent already. So ideally, I think we should enable mob by default and the threshold maybe 4MB? And if it does not write the big data directly to WAL, then we could make {{AsyncFSWAL}} default.

Thanks. [~anoopsamjohn]

> Make AsyncFSWAL as our default WAL
> ----------------------------------
>
>                 Key: HBASE-15536
>                 URL: https://issues.apache.org/jira/browse/HBASE-15536
>             Project: HBase
>          Issue Type: Sub-task
>          Components: wal
>    Affects Versions: 2.0.0
>            Reporter: Duo Zhang
>            Assignee: Duo Zhang
>             Fix For: 2.0.0
>
>         Attachments: HBASE-15536-v1.patch, HBASE-15536-v2.patch, HBASE-15536-v3.patch, HBASE-15536-v4.patch, HBASE-15536-v5.patch, HBASE-15536.patch
>
>
> As it should be predicated on passing basic cluster ITBLL



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)