You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Eric Barault (Jira)" <ji...@apache.org> on 2022/07/04 10:03:00 UTC
[jira] [Updated] (SPARK-39619) PrometheusServlet: add "TYPE" comment to exposed metrics
[ https://issues.apache.org/jira/browse/SPARK-39619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Barault updated SPARK-39619:
---------------------------------
Description:
The PrometheusServlet sink does not include the usual comments when exposing the metrics in the prometheus format
e.g. `# TYPE nginx_ingress_controller_ingress_upstream_latency_seconds summary`
which prevents some client/integrations that depend on them to assess the metric type to work properly.
For example the AWS cloudwatch agent prometheus plugin attempts to get the metric type from the TYPE comment and considers any metric with no type as unsuported and hence drops it.
As a result the cloudwatch agent prometheus drops all the metrics exposed by the PrometheusServlet.
[https://github.com/aws/amazon-cloudwatch-agent/blob/1f654cf69c1269073673ba2f636738c556248a31/plugins/inputs/prometheus_scraper/metrics_type_handler.go#L190]
This would be solved by adding the TYPE comments to the metrics exposed by the PrometheusServlet sink.
_*references:*_
- [https://dzlab.github.io/bigdata/2020/07/03/spark3-monitoring-1/]
- [https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/metrics/sink/PrometheusServlet.scala]
- [https://github.com/prometheus/docs/blob/main/content/docs/instrumenting/exposition_formats.md]
was:
The PrometheusServlet sink does not include the usual comments when exposing the metrics in the prometheus format
e.g. `# TYPE nginx_ingress_controller_ingress_upstream_latency_seconds summary`
which prevents some client/integrations that depend on them to assess the metric type to work properly.
For example the AWS cloudwatch agent prometheus plugin attempts to get the metric type from the TYPE comment and considers any metric with no type as unsuported and hence drops it.
As a result the cloudwatch agent prometheus drops all the metrics exposed by the PrometheusServlet.
[https://github.com/aws/amazon-cloudwatch-agent/blob/1f654cf69c1269073673ba2f636738c556248a31/plugins/inputs/prometheus_scraper/metrics_type_handler.go#L190]
This would be solved by adding the TYPE comments to the metrics exposed by the PrometheusServlet sink.
_*references:*_
- [https://dzlab.github.io/bigdata/2020/07/03/spark3-monitoring-1/]
- [https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/metrics/sink/PrometheusServlet.scala]
> PrometheusServlet: add "TYPE" comment to exposed metrics
> --------------------------------------------------------
>
> Key: SPARK-39619
> URL: https://issues.apache.org/jira/browse/SPARK-39619
> Project: Spark
> Issue Type: Improvement
> Components: Input/Output
> Affects Versions: 3.0.0, 3.1.0, 3.2.0, 3.3.0
> Reporter: Eric Barault
> Priority: Major
>
> The PrometheusServlet sink does not include the usual comments when exposing the metrics in the prometheus format
> e.g. `# TYPE nginx_ingress_controller_ingress_upstream_latency_seconds summary`
> which prevents some client/integrations that depend on them to assess the metric type to work properly.
> For example the AWS cloudwatch agent prometheus plugin attempts to get the metric type from the TYPE comment and considers any metric with no type as unsuported and hence drops it.
> As a result the cloudwatch agent prometheus drops all the metrics exposed by the PrometheusServlet.
> [https://github.com/aws/amazon-cloudwatch-agent/blob/1f654cf69c1269073673ba2f636738c556248a31/plugins/inputs/prometheus_scraper/metrics_type_handler.go#L190]
>
> This would be solved by adding the TYPE comments to the metrics exposed by the PrometheusServlet sink.
>
> _*references:*_
> - [https://dzlab.github.io/bigdata/2020/07/03/spark3-monitoring-1/]
> - [https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/metrics/sink/PrometheusServlet.scala]
> - [https://github.com/prometheus/docs/blob/main/content/docs/instrumenting/exposition_formats.md]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org