You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Yu Li (JIRA)" <ji...@apache.org> on 2018/04/20 10:39:00 UTC
[jira] [Commented] (HBASE-19024) Configurable default durability
for synchronous WAL
[ https://issues.apache.org/jira/browse/HBASE-19024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445592#comment-16445592 ]
Yu Li commented on HBASE-19024:
-------------------------------
Sorry to be late for the party, just noticed this one.
Skimmed the patch, it seems now we're mixing the fsync(hsync) sequence id with sync(hflush), and if a sync call with higher sequenceId got executed before a fsync call (due to multiple syncRunners), the fsync call is actually written in a hflush way rather than expected. This issue was proposed and discussed back in HBASE-5954 around [this comment|https://issues.apache.org/jira/browse/HBASE-5954?focusedCommentId=14222296&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14222296], so are we ignoring this intentionally or by mistake? Worth a double check? Thanks. [~stack] [~apurtell] [~anoop.hbase]
> Configurable default durability for synchronous WAL
> ---------------------------------------------------
>
> Key: HBASE-19024
> URL: https://issues.apache.org/jira/browse/HBASE-19024
> Project: HBase
> Issue Type: Improvement
> Components: wal
> Reporter: Vikas Vishwakarma
> Assignee: Harshal Jain
> Priority: Critical
> Fix For: 3.0.0, 2.1.0, 1.5.0
>
> Attachments: HBASE-19024-master.v10.patch, HBASE-19024.branch-1.2.001.patch, HBASE-19024.branch-1.2.002.patch, HBASE-19024.branch-1.2.003.patch, HBASE-19024.branch-1.2.004.patch, HBASE-19024.branch-1.2.005.patch, branch-1.branch-1.patch, branch-1.v1.branch-1.patch, master.patch, master.v2.patch, master.v3.patch, master.v5.patch, master.v5.patch, master.v6.patch, master.v9.patch
>
>
> At present we do not have an option to hsync WAL edits to the disk for better durability. In our local tests we see 10-15% latency impact of using hsync instead of hflush which is not very high.
> We should have a configurable option to hysnc WAL edits instead of just sync/hflush which will call the corresponding API on the hadoop side. Currently HBase handles both SYNC_WAL and FSYNC_WAL as the same calling FSDataOutputStream sync/hflush on the hadoop side. This can be modified to let FSYNC_WAL call hsync on the hadoop side instead of sync/hflush. We can keep the default value to sync as the current behavior and hsync can be enabled based on explicit configuration.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)