[GitHub] [accumulo] EdColeman commented on pull request #2188: WIP: Add micrometer metrics implementation

[GitBox <gi...@apache.org>]

EdColeman commented on pull request #2188:
URL: https://github.com/apache/accumulo/pull/2188#issuecomment-883595649


   A follow task may be to implement a bridge between micrometer and the hadoop metrics system so that, if a hadoop exporter was present, it would read metrics from micrometer and then publish to hadoop configured sinks.
   
   The basic concept would be to find / extend a micrometer registry (see https://micrometer.io/docs/guide/customMeterRegistry) that would instantiate a hadoop metrics system (see System design - https://blog.cloudera.com/what-is-hadoop-metrics2/ ) The custom micrometer registry would function as the "metrics sources" and the hadoop metrics system would configure the sinks by reading the hadoop.metrics.xxx.properties file.  So when the hadoop metrics system "read" metrics from the sources, the values would be obtained via the micrometer metrics registry.  The goal would be that the Accumulo would only have micrometer Meters (gauges / counters,...) and if a hadoop metrics exporter was configured, it would read from the micrometer registry to get those values.  
   
   I'm not sure that the hadoop metrics would be compatible with a StepMeterRegistry - it might require the more general MeterRegistry.  Using a StepMeterRegistry might be enough to figure out some of the concepts - and then find an existing registry (etsy statsd, datadog,...) that matches the hadoop metrics system dimensionality (hierarchical / dimensional), rate aggregation (client / server) and publication (pull / push)  My impression is that hadoop would be hierarchical, client and pull) but that's making some guesses and needs to be confirmed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org