Replication Latency and Metrics

[Noe Detore <nd...@minerkasch.com>]

Hello,

I notice there are a few metrics coming out for replication:
FilesPendingReplicationNumOps
FilesPendingReplicationAvgTime
MasterReplication.NumPeersNumOps
MasterReplication.NumPeersAvgTime
MaxReplicationThreadsNumOps
MaxReplicationThreadsAvgTime

What can be learned from these?

Is there a way to know or measure the latency of from when a record is
written to when it is replicated?

Thank you
Noe

Re: Replication Latency and Metrics

[Josh Elser <jo...@gmail.com>]

No, Accumulo tracks replication at the (write-ahead log) file level, not 
at the update level.

For the names you've listed, you should also see values for each. I am 
assuming that you did not copy all of the exposed metrics. I believe the 
NumOps and AvgTime values are information generated by the metrics 
framework itself about how many times each metric was polled and the 
amount of time it took to compute the value.

For a description on what each value actually measures, please read [1] 
and reply back with specific questions about what you don't understand.

[1] 
https://github.com/apache/accumulo/blob/e900e67425d950bd4c0c5288a6270d7b362ac458/server/master/src/main/java/org/apache/accumulo/master/metrics/ReplicationMetricsMBean.java

Noe Detore wrote:
> Hello,
>
> I notice there are a few metrics coming out for replication:
> FilesPendingReplicationNumOps
> FilesPendingReplicationAvgTime
> MasterReplication.NumPeersNumOps
> MasterReplication.NumPeersAvgTime
> MaxReplicationThreadsNumOps
> MaxReplicationThreadsAvgTime
>
> What can be learned from these?
>
> Is there a way to know or measure the latency of from when a record is
> written to when it is replicated?
>
> Thank you
> Noe