[jira] [Created] (SPARK-38117) Executor metrics are missing on prometheus sink

["zoli (Jira)" <ji...@apache.org>]

zoli created SPARK-38117:
----------------------------

             Summary: Executor metrics are missing on prometheus sink
                 Key: SPARK-38117
                 URL: https://issues.apache.org/jira/browse/SPARK-38117
             Project: Spark
          Issue Type: Bug
          Components: Spark Core
    Affects Versions: 3.1.2
         Environment: versions: Spark3.1.2, K8s v19
            Reporter: zoli


Setting up prometheus sink in this way:
-c spark.ui.prometheus.enabled=true
-c spark.executor.processTreeMetrics.enabled=true
-c spark.metrics.conf=/spark/conf/metric.properties
{{}}

{_}*metric.properties:*{_}{{{}{}}}
*.sink.prometheusServlet.class=org.apache.spark.metrics.sink.PrometheusServlet
*.sink.prometheusServlet.path=/metrics/prometheus
Result:

Both of these endpoints have some metrics
<driver-ip>:4040/metrics/prometheus        
<driver-ip>:4040/metrics/executors/prometheus
{{But the executor one}} misses metrics under the executor namespace described here: [https://spark.apache.org/docs/3.1.2/monitoring.html#component-instance--executor]
So everything is missing from {{bytesRead.count}} to {{threadpool.startedTasks}}

There are neither error nor warn level entries in the driver/executor logs.

By changing to ConsoleSink  I can see all the necessary metrics:
*.sink.console.class=org.apache.spark.metrics.sink.ConsoleSink
*.sink.console.period=10
*.sink.console.unit=seconds
{{ }}
{{Something is wrong with the spark-prometheus integration}}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org