[jira] [Created] (FLINK-38603) FLIP-553 [Metrics] Robust OTel gRPC metric exporter

["Aleksandr Iushmanov (Jira)" <ji...@apache.org>]

Aleksandr Iushmanov created FLINK-38603:
-------------------------------------------

             Summary: FLIP-553 [Metrics] Robust OTel gRPC metric exporter
                 Key: FLINK-38603
                 URL: https://issues.apache.org/jira/browse/FLINK-38603
             Project: Flink
          Issue Type: New Feature
            Reporter: Aleksandr Iushmanov


Jobs with large number of tasks and operator can result in significant payload size. Open Telemetry exporter gRPC calls can be rejected due to too large payload leading to loss of exported metric data in production environments. Existing implementation presents the following limitations:
 # While OTel exporter allows to enable gZip compression for requests, it is not available in Flink Configuration.

 # All metric data is exported in a single gRPC call, with no pagination which impacts complex enough jobs.

 # Metric attributes can be unexpectedly large, for example task_name attribute can contain hundreds of operator names inside and be present in multiple metrics having a significant impact on overall payload.

 

[Link to FLIP-553 page|https://cwiki.apache.org/confluence/display/FLINK/FLIP-553%3A+Robust+OTel+gRPC+metric+exporter]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)