You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Eric Barault (Jira)" <ji...@apache.org> on 2022/06/28 08:27:00 UTC

[jira] [Updated] (SPARK-39619) PrometheusServlet: add "TYPE" comment to exposed metrics

|  ![](cid:jira-generated-image-avatar-c59abb2d-897b-4e13-a495-53ba43f21f73) |
[Eric
Barault](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=ebarault)
**updated** an issue  
---|---  
|  
---  
|  [Spark](https://issues.apache.org/jira/browse/SPARK) /
[![Improvement](cid:jira-generated-image-
avatar-6e089a99-8dd9-42ee-8cb1-385dfad786fa)](https://issues.apache.org/jira/browse/SPARK-39619)
[SPARK-39619](https://issues.apache.org/jira/browse/SPARK-39619)  
---  
[PrometheusServlet: add "TYPE" comment to exposed
metrics](https://issues.apache.org/jira/browse/SPARK-39619)  
| Change By: |  [Eric
Barault](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=ebarault)  
---|---  
|  The PrometheusServlet sink does not include the usual comments when
exposing the metrics in the prometheus format  
  
e.g. `# TYPE nginx_ingress_controller_ingress_upstream_latency_seconds
summary`  
  
which prevents some client/integrations that depend on them to assess the
metric type to work properly.  
  
For example the AWS cloudwatch agent prometheus plugin attempts to get the
metric type from the TYPE comment and considers any metric with no type as
unsuported and hence drops it.  
  
As a result the cloudwatch agent prometheus drops all the metrics exposer
exposed by the PrometheusServlet.  
  
[ https://github.com/aws/amazon-cloudwatch-
agent/blob/1f654cf69c1269073673ba2f636738c556248a31/plugins/inputs/prometheus_scraper/metrics_type_handler.go#L190
]  
  
  
  
This would be solved by adding the TYPE comments to the metrics exposed by the
PrometheusServlet sink.  
  
  
  
_*references:*_  
  
- [ https://dzlab.github.io/bigdata/2020/07/03/spark3-monitoring-1/ ]   
  
- [ https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/metrics/sink/PrometheusServlet.scala ]  
---  
|  |  [ ![Add Comment](cid:jira-generated-image-static-comment-
icon-4a614285-14a2-4e03-9330-41dec7d1761b)
](https://issues.apache.org/jira/browse/SPARK-39619#add-comment "Add Comment")
|  [Add Comment](https://issues.apache.org/jira/browse/SPARK-39619#add-comment
"Add Comment")  
---|---  
  
|  This message was sent by Atlassian Jira (v8.20.10#820010-sha1:ace47f9) |  |
![Atlassian logo](https://issues.apache.org/jira/images/mail/atlassian-email-
logo.png)  
---