Region movements and HBase replication

[techbuddy <te...@gmail.com>]

Hi,

So I was wondering how to surmount a possible out-of-order replication of
edits belonging to a particular region in the event the latter gets moved
from one region server to another. 

The memstores in the region would get flushed prior to the movement of the
region from Region server RS1 to RS2, but if the replication source(s) on
RS1 is(are) slow at shipping the edits, it may so happen that there are
edits pending replication in the WAL of RS1, while the region is getting
fresh edits to the WAL on RS2, after being made available on RS2.

And since the replication sources on the two RSes are working independently,
it's possible that the edits to a given region (and therefore a given rows)
get processed out of order. 

Do you guys agree with the scenario. If so, is there any plan to fix this.
It could be largely a non-issue as long as the edits are touching different
rows in the region, but could be a stark one when the edits touch the same
row.

-thanks
SB



--
View this message in context: http://apache-hbase.679495.n3.nabble.com/Region-movements-and-HBase-replication-tp4052700.html
Sent from the HBase Developer mailing list archive at Nabble.com.

Re: Region movements and HBase replication

[Jeffrey Zhong <jz...@hortonworks.com>]

To my knowledge, the scenario you mentioned below could happen. The
similar issue could happen in another scenario: replication region server
failover case because failover replication waits for RS fail over
completes and then replicate failed over edits. By then, the new RS could
replicate edits of the same region while logs in fail over queue are being
replicated.

Fortunately the issue can only happen when a client sends same version
updates(means exact same row key plus same timestamp). If you don't
provide timestamp in your client code then I think you're fine. The issue
isn't trivial to fix and basically requires storing origin sequence number
along with KVs till those race time window pass.

Thanks.

On 11/11/13 1:59 PM, "techbuddy" <te...@gmail.com> wrote:

>Hi,
>
>So I was wondering how to surmount a possible out-of-order replication of
>edits belonging to a particular region in the event the latter gets moved
>from one region server to another.
>
>The memstores in the region would get flushed prior to the movement of the
>region from Region server RS1 to RS2, but if the replication source(s) on
>RS1 is(are) slow at shipping the edits, it may so happen that there are
>edits pending replication in the WAL of RS1, while the region is getting
>fresh edits to the WAL on RS2, after being made available on RS2.
>
>And since the replication sources on the two RSes are working
>independently,
>it's possible that the edits to a given region (and therefore a given
>rows)
>get processed out of order.
>
>Do you guys agree with the scenario. If so, is there any plan to fix this.
>It could be largely a non-issue as long as the edits are touching
>different
>rows in the region, but could be a stark one when the edits touch the same
>row.
>
>-thanks
>SB
>
>
>
>--
>View this message in context:
>http://apache-hbase.679495.n3.nabble.com/Region-movements-and-HBase-replic
>ation-tp4052700.html
>Sent from the HBase Developer mailing list archive at Nabble.com.



-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.