pre and postReplicateLogEntries from RegionServerObserver

[deepw <ge...@gmail.com>]

We are trying to use the RegionServerObserver to track the current status of
Replication [for a specific set of tables only]. Based on some experiments I
see that the preReplicateLogEntries and postReplicateLogEntries gets called
on the slave cluster region server but not on the Master. The same
coprocessor is loaded on both master and slave cluster. We see the "start"
and the "stop" [from RegionServerObserver] functions being called correctly
on the Master but not the replication related functions. 
Is this expected or are we missing something here. 

Thanks
Deepak. 



--
View this message in context: http://apache-hbase.679495.n3.nabble.com/pre-and-postReplicateLogEntries-from-RegionServerObserver-tp4085153.html
Sent from the HBase User mailing list archive at Nabble.com.

Re: pre and postReplicateLogEntries from RegionServerObserver

[deepw <ge...@gmail.com>]

Thank You Ted for your super fast reply.
What I am trying to determine is the timestamp [writeTime from the WALKey]
from the Master cluster that has been successfully replicated to the Slave.
My intention is to compare this time to a certain wall clock time of
interest to guarantee that all entries written before this time stamp have
been successfully replicated to the slave cluster.

We cannot use the existing Replication Metrics [like ageOfLastShippedOp]
because there could be other tables being replicated between the two
clusters which are not of interest. My approach is if the timestamp of the
last replicated entry is ahead of the time of interest, then "all" entries
have been replicated so the entries of interest must also have been
replicated.

Please let me know if I could provide more details on the use case. I
welcome your comments on the approach as well.

Thanks again for the help.

Regards
Deepak .

On Thu, Dec 15, 2016 at 7:04 PM, Ted Yu-3 [via Apache HBase] <
ml-node+s679495n4085155h41@n3.nabble.com> wrote:

> bq. preReplicateLogEntries and postReplicateLogEntries gets called on the
> slave cluster region server
>
> This is by design.
> These two hooks are around ReplicationSinkService#replicateLogEntries().
> ReplicationSinkService represents the sink.
>
> Can you tell us what you need to know on the source cluster ?
>
> On Thu, Dec 15, 2016 at 6:52 PM, deepw <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=4085155&i=0>> wrote:
>
> > We are trying to use the RegionServerObserver to track the current
> status
> > of
> > Replication [for a specific set of tables only]. Based on some
> experiments
> > I
> > see that the preReplicateLogEntries and postReplicateLogEntries gets
> called
> > on the slave cluster region server but not on the Master. The same
> > coprocessor is loaded on both master and slave cluster. We see the
> "start"
> > and the "stop" [from RegionServerObserver] functions being called
> correctly
> > on the Master but not the replication related functions.
> > Is this expected or are we missing something here.
> >
> > Thanks
> > Deepak.
> >
> >
> >
> > --
> > View this message in context: http://apache-hbase.679495.n3.
> > nabble.com/pre-and-postReplicateLogEntries-from-RegionServerObserver-
> > tp4085153.html
> > Sent from the HBase User mailing list archive at Nabble.com.
> >
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-hbase.679495.n3.nabble.com/pre-and-
> postReplicateLogEntries-from-RegionServerObserver-tp4085153p4085155.html
> To unsubscribe from pre and postReplicateLogEntries from
> RegionServerObserver, click here
> <http://apache-hbase.679495.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4085153&code=Z2VtaW5pLmRlZXBha0BnbWFpbC5jb218NDA4NTE1M3w1NjE0NTc5NDg=>
> .
> NAML
> <http://apache-hbase.679495.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>




--
View this message in context: http://apache-hbase.679495.n3.nabble.com/pre-and-postReplicateLogEntries-from-RegionServerObserver-tp4085153p4085157.html
Sent from the HBase User mailing list archive at Nabble.com.

Re: pre and postReplicateLogEntries from RegionServerObserver

[Ted Yu <yu...@gmail.com>]

bq. preReplicateLogEntries and postReplicateLogEntries gets called on the
slave cluster region server

This is by design.
These two hooks are around ReplicationSinkService#replicateLogEntries().
ReplicationSinkService represents the sink.

Can you tell us what you need to know on the source cluster ?

On Thu, Dec 15, 2016 at 6:52 PM, deepw <ge...@gmail.com> wrote:

> We are trying to use the RegionServerObserver to track the current status
> of
> Replication [for a specific set of tables only]. Based on some experiments
> I
> see that the preReplicateLogEntries and postReplicateLogEntries gets called
> on the slave cluster region server but not on the Master. The same
> coprocessor is loaded on both master and slave cluster. We see the "start"
> and the "stop" [from RegionServerObserver] functions being called correctly
> on the Master but not the replication related functions.
> Is this expected or are we missing something here.
>
> Thanks
> Deepak.
>
>
>
> --
> View this message in context: http://apache-hbase.679495.n3.
> nabble.com/pre-and-postReplicateLogEntries-from-RegionServerObserver-
> tp4085153.html
> Sent from the HBase User mailing list archive at Nabble.com.
>