[jira] [Commented] (HBASE-20471) Recheck the design and implementation of FSYNC_WAL durability for WAL

["Yu Li (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-20471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16446692#comment-16446692 ] 

Yu Li commented on HBASE-20471:
-------------------------------

Some of my thoughts:

1. Only allow setting {{SYNC_WAL}} through {{Mutation#setDurability}} API
    * We can separate FSYNC_WAL from SYNC_WAL writes only if we flush the mutations one by one, but actually we're grouping (and we should, for the sake of performance) the writes through disruptor mechanism, which makes it possible that a mutation with SYNC_WAL durability is actually fsync'ed, vice versa.
    * To keep backward compatibility, if user set FSYNC_WAL, we should change it to SYNC_WAL in {{Mutation#setDurability}}
    * We should add some document about this change in our ref-guide.

2. Instead, we allow user to set the cluster-level way of sync through hbase-site.xml, hflush or hsync.

3. In the future we may allow user to use a dedicated WAL per table/CF and set its sync mode through table/CF descriptor.

> Recheck the design and implementation of FSYNC_WAL durability for WAL
> ---------------------------------------------------------------------
>
>                 Key: HBASE-20471
>                 URL: https://issues.apache.org/jira/browse/HBASE-20471
>             Project: HBase
>          Issue Type: Task
>            Reporter: Yu Li
>            Priority: Major
>
> This is something derived from discussion in HBASE-19024 around [this comment|https://issues.apache.org/jira/browse/HBASE-19024?focusedCommentId=16445592&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16445592]
> We have been supplying user the API to set durability per mutation for a long time, by design the SYNC_WAL durability to call {{FSDataOutputStream#hflush}} and FSYNC_WAL {{FSDataOutputStream#hsync}}, while in implementation we have been calling hflush for FSYNC_WAL also until HBASE-19024. Although HBASE-19024 tried to fix the syntax with good willing, the implementation there cannot assure the FSYNC_WAL edits are truly hsync'ed due to the disruptor mechanism used in WAL implementation. Here in this JIRA we aim to have more discussion and give it a complete solution.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)