[jira] [Created] (HBASE-22660) Precise end to end tracking of cross cluster replication latency

["Andrew Purtell (JIRA)" <ji...@apache.org>]

Andrew Purtell created HBASE-22660:
--------------------------------------

             Summary: Precise end to end tracking of cross cluster replication latency
                 Key: HBASE-22660
                 URL: https://issues.apache.org/jira/browse/HBASE-22660
             Project: HBase
          Issue Type: New Feature
            Reporter: Andrew Purtell


ageOfLastShippedOp tracks replication latency forward from the point where a source process tailing a WAL has found an edit to ship. This is not an end to end measure.

To achieve a holistic end to end measure we should have an active process that periodically injects sentinel values at commit time adjacent to the WALedits carrying application data at the source and records when they are finally processed at the sink, using a timestamp embedded in the sentinel to measure true end to end latency for the adjacent commit. This could be done for a configurable (and small) percentage of commits so would give a probabilistic measure with confidence controlled by sample rate. It should be done this way rather than by passively sampling cell timestamps because cell timestamps can be set by the user and may not correspond to wall clock time.

We could introduce a new type of synthetic WALedit, a new global metric, and because the adjacent commit from which we build the sentinel contains table information we could track that too and add a per table metric. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)