You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Lars Hofhansl (JIRA)" <ji...@apache.org> on 2015/05/04 02:48:06 UTC

[jira] [Commented] (HBASE-12988) [Replication]Parallel apply edits on row-level

    [ https://issues.apache.org/jira/browse/HBASE-12988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526121#comment-14526121 ] 

Lars Hofhansl commented on HBASE-12988:
---------------------------------------

Thinking on this again, it's important we do this. Otherwise replication is slow, especially across high latency links across data centers.
We can significantly cut this down but by having multiple smaller chunks in flight to multiple target region server instead of only a single large chunk to exactly one region server at a time.

The grouping by row should be easy to do to avoid any weirdness around ordering of deletes/put for the same row.

The place to do this might be HBaseInterClusterReplicationEndpoint.replication or its caller.
There we get one (large) chunk, and send it to exactly one peer region server. We can split the chunk up into N parts here and farm it to N region servers in parallel, if all sub chunks are success the entire large chunk is successful. We can retry individual chunks a few time and then fail the larger chunk if that failed.


> [Replication]Parallel apply edits on row-level
> ----------------------------------------------
>
>                 Key: HBASE-12988
>                 URL: https://issues.apache.org/jira/browse/HBASE-12988
>             Project: HBase
>          Issue Type: Improvement
>          Components: Replication
>            Reporter: hongyu bi
>            Assignee: hongyu bi
>
> we can apply  edits to slave cluster in parallel on table-level to speed up replication .
> update : per conversation blow , it's better to apply edits on row-level in parallel



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)