[jira] [Commented] (HBASE-10504) Define Replication Interface

["Enis Soztutar (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-10504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13992995#comment-13992995 ] 

Enis Soztutar commented on HBASE-10504:
---------------------------------------

I have a very basic prototype patch that we can consider as a direction for this. The idea is to break up ReplicationSource into two parts, the part that tails the WAL and listens to the replication queue, and the part that talks to the ReplicationSink in the peer cluster to ship the edits. The new interface "ReplicationConsumer" (we can change the names) is the plugin point. 

There is still one ReplicationSource per peer in every RS. The ReplicationSource tails the logs for the peer, and holds a ReplicationConsumer pointer. It calls ReplicationConsumer.shipEdits() method for actual shipping logic. A default HBaseClusterReplicator implements ReplicationConsumer and talks to the ReplicationSink on the other cluster etc. 

For each peer, there is now a class implementing ReplicationConsumer, and possibly some configuration. This allows one to implement a ReplicationConsumer for SOLR (or any other data store) and directly plug it in to the RS without having to do a proxy layer and mocking zookeeper / RS RPC etc. 

Let me know what you think. Early feedback will help shape up the patch. 





> Define Replication Interface
> ----------------------------
>
>                 Key: HBASE-10504
>                 URL: https://issues.apache.org/jira/browse/HBASE-10504
>             Project: HBase
>          Issue Type: Task
>            Reporter: stack
>            Assignee: stack
>            Priority: Blocker
>             Fix For: 0.99.0
>
>         Attachments: hbase-10504_wip1.patch
>
>
> HBase has replication.  Fellas have been hijacking the replication apis to do all kinds of perverse stuff like indexing hbase content (hbase-indexer https://github.com/NGDATA/hbase-indexer) and our [~toffer] just showed up w/ overrides that replicate via an alternate channel (over a secure thrift channel between dcs over on HBASE-9360).  This issue is about surfacing these APIs as public with guarantees to downstreamers similar to those we have on our public client-facing APIs (and so we don't break them for downstreamers).
> Any input [~phunt] or [~gabriel.reid] or [~toffer]?
> Thanks.
>  



--
This message was sent by Atlassian JIRA
(v6.2#6252)