[jira] [Comment Edited] (HBASE-14004) [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin

["Heng Chen (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-14004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15046414#comment-15046414 ] 

Heng Chen edited comment on HBASE-14004 at 12/8/15 6:01 AM:
------------------------------------------------------------

Why we need 'where to end' when recover replication?   We just need to start replicate entries where replication were broken when RS failed, right? If you concern we may read 'unhsynced wal entries',  we could close relates WAL first before recovery if it is still open. right?


was (Author: chenheng):
Why we need 'where to end' when recover replication?   We just need to start replicate entries where replication were broken when RS failed, right?

> [Replication] Inconsistency between Memstore and WAL may result in data in remote cluster that is not in the origin
> -------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-14004
>                 URL: https://issues.apache.org/jira/browse/HBASE-14004
>             Project: HBase
>          Issue Type: Bug
>          Components: regionserver
>            Reporter: He Liangliang
>            Priority: Critical
>              Labels: replication, wal
>
> Looks like the current write path can cause inconsistency between memstore/hfile and WAL which cause the slave cluster has more data than the master cluster.
> The simplified write path looks like:
> 1. insert record into Memstore
> 2. write record to WAL
> 3. sync WAL
> 4. rollback Memstore if 3 fails
> It's possible that the HDFS sync RPC call fails, but the data is already  (may partially) transported to the DNs which finally get persisted. As a result, the handler will rollback the Memstore and the later flushed HFile will also skip this record.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)