[GitHub] [pinot] gortiz commented on issue #8762: Allow plugins to register and report metrics

[GitBox <gi...@apache.org>]

gortiz commented on issue #8762:
URL: https://github.com/apache/pinot/issues/8762#issuecomment-1350577858

   Do we actually need this? Metrics are not that different than logging. We do not have a logging api, we use slf4j. If some dependency we have uses log4j to log, then we can simply use the slf4j bridge to have them.
   
   Going back to metrics, each plugin could include a dependency to dropwizard/micrometer and register their metrics as JMX beans. Then Prometheus collector will see them classified by their tags in the same way that log4j2.xml files can see logs classified by their classes/packages.
   
   In my opinion, with the metrics registry we are adding a bunch of indirections that instead of helping us to have a generic system that is easier to change from one metric library to the other, is making that change more difficult. In fact we are still using yammer when the community is already stopping to use the yammer successor, dropwizard.
   
   The obvious con to have plugins directly using a metric library is that all plugins must use a compatible library version. That could also be said when talking about logging. And that is why mature logging and metric libraries maintain a good backward compatibility.
   
   Another thing to consider is the metrics of third party libraries. Again we can do a comparison with logging. These third party libraries log with slf4j or log4j and we collect their logs with log4j. The same should happen with metrics. They can use mostly whatever they want to register their metrics in the JMX context and we can use Prometheus to collect them. Obviously there are more libraries that register logs than libraries than register metrics, but there are libraries that do that. For example Zookeeper, Helix and Kafka clients expose their own metrics.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org