You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "Fabian Paul (Jira)" <ji...@apache.org> on 2020/03/11 16:23:00 UTC

[jira] [Created] (FLINK-16553) KafkaFetcher topic/partition metrics

Fabian Paul created FLINK-16553:
-----------------------------------

             Summary: KafkaFetcher topic/partition metrics
                 Key: FLINK-16553
                 URL: https://issues.apache.org/jira/browse/FLINK-16553
             Project: Flink
          Issue Type: Improvement
          Components: Connectors / Kafka, Runtime / Metrics
            Reporter: Fabian Paul


When using the Kafka universal connector, currently not all KafkaFetcher metrics ([link| https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetcherMetricsRegistry.java] which are exposed through the KafkaConsumer are accessible within the Flink metrics system.

Especially, all metrics which are related to topics and partitions are not available. The KafkaConsumer internally only registers those metrics after it has fetched some records.

Unfortunately, at the moment Flink only checks the available metrics right after the initialization of the KafkaConsumer when no records are polled, yet.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)