[GitHub] [arrow-ballista] thinkharderdev commented on a diff in pull request #504: Prometheus metrics

[GitBox <gi...@apache.org>]

thinkharderdev commented on code in PR #504:
URL: https://github.com/apache/arrow-ballista/pull/504#discussion_r1016758023


##########
ballista/scheduler/src/metrics/prometheus.rs:
##########
@@ -0,0 +1,141 @@
+use crate::metrics::SchedulerMetricsCollector;
+use ballista_core::error::{BallistaError, Result};
+use hyper::header::CONTENT_TYPE;
+use prometheus::{
+    register_counter_with_registry, register_gauge_with_registry,
+    register_histogram_with_registry, Counter, Gauge, Histogram, Registry,
+};
+use prometheus::{Encoder, TextEncoder};
+
+use warp::Reply;
+
+pub struct PrometheusMetricsCollector {
+    execution_time: Histogram,
+    planning_time: Histogram,
+    failed: Counter,
+    cancelled: Counter,
+    completed: Counter,
+    submitted: Counter,
+    pending_queue_size: Gauge,
+}
+
+impl PrometheusMetricsCollector {
+    pub fn new(registry: &Registry) -> Result<Self> {
+        let execution_time = register_histogram_with_registry!(
+            "query_time_seconds",
+            "Histogram of query execution time in seconds",
+            vec![0.5_f64, 1_f64, 5_f64, 30_f64, 60_f64],
+            registry
+        )
+        .map_err(|e| {
+            BallistaError::Internal(format!("Error registering metric: {:?}", e))
+        })?;
+
+        let planning_time = register_histogram_with_registry!(
+            "planning_time_ms",
+            "Histogram of query planning time in milliseconds",
+            vec![1.0_f64, 5.0_f64, 25.0_f64, 100.0_f64, 500.0_f64],
+            registry
+        )
+        .map_err(|e| {
+            BallistaError::Internal(format!("Error registering metric: {:?}", e))
+        })?;

Review Comment:
   These histograms are hard to generalize since use cases could vary wildly. We can either:
   1. Make these configurable
   2. Try and choose sensible defaults and if someone needs to customize then they can implement a custom collector



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org