You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Yunhui Han <la...@gmail.com> on 2020/09/15 07:44:44 UTC
Kafka Consumer Metrics For Help
Hi, all
I use influxdb to monitor flink metrics. However, I am confused with the
metric taskmanager job_task_operator_KafkaConsumer_count. What does it
mean? Is it OK for that the value is around 150 ?
[image: image.png]
Your Sincerely
Re: Kafka Consumer Metrics For Help
Posted by Dawid Wysakowicz <dw...@apache.org>.
As I said the infix "KafkaConsumer" means it is a metric forwarded from
Kafka client itself. (most probably there are just a few exceptions).
This one in particular is the count of registered metrics[1].
metricName("count", "kafka-metrics-count", "total number of registered
metrics")
Best,
Dawid
[1] https://issues.apache.org/jira/browse/KAFKA-2668
On 16/09/2020 03:33, Yunhui Han wrote:
> Hi，Dawid
> Thank you for your response.
> The version of Flink is flink-1.10.1, and the Kafka version is 0.10.
> The connector version is flink-connector-kafka-0.10_2.11.
> This metric is generated by flink itself. I did not register any
> custom metrics.
> Best
>
>
> On Tue, Sep 15, 2020 at 8:16 PM Dawid Wysakowicz
> <dwysakowicz@apache.org <ma...@apache.org>> wrote:
>
> Hi,
>
> Which version of Flink, Kafka and Kafka connector are you using?
> The KafkaConsumer infix means it is most probably a metric
> forwarded from the Kafka client. (I could not find one, named like
> that, so a concrete versions of different components could help).
>
> Do you register any custom metrics in a TimestampAssigner or
> WatermarkGenerator?
>
> Best,
>
> Dawid
>
> On 15/09/2020 09:44, Yunhui Han wrote:
>> Hi, all
>>
>> I use influxdb to monitor flink metrics. However, I am confused
>> with the metric taskmanager
>> job_task_operator_KafkaConsumer_count. What does it mean? Is it
>> OK for that the value is around 150 ?
>> image.png
>> Your Sincerely
>
Re: Kafka Consumer Metrics For Help
Posted by Yunhui Han <la...@gmail.com>.
Hi，Dawid
Thank you for your response.
The version of Flink is flink-1.10.1, and the Kafka version is 0.10. The
connector version is flink-connector-kafka-0.10_2.11.
This metric is generated by flink itself. I did not register any custom
metrics.
Best
On Tue, Sep 15, 2020 at 8:16 PM Dawid Wysakowicz <dw...@apache.org>
wrote:
> Hi,
>
> Which version of Flink, Kafka and Kafka connector are you using? The
> KafkaConsumer infix means it is most probably a metric forwarded from the
> Kafka client. (I could not find one, named like that, so a concrete
> versions of different components could help).
>
> Do you register any custom metrics in a TimestampAssigner or
> WatermarkGenerator?
>
> Best,
>
> Dawid
> On 15/09/2020 09:44, Yunhui Han wrote:
>
> Hi, all
>
> I use influxdb to monitor flink metrics. However, I am confused with the
> metric taskmanager job_task_operator_KafkaConsumer_count. What does it
> mean? Is it OK for that the value is around 150 ?
> [image: image.png]
> Your Sincerely
>
>
Re: Kafka Consumer Metrics For Help
Posted by Dawid Wysakowicz <dw...@apache.org>.
Hi,
Which version of Flink, Kafka and Kafka connector are you using? The
KafkaConsumer infix means it is most probably a metric forwarded from
the Kafka client. (I could not find one, named like that, so a concrete
versions of different components could help).
Do you register any custom metrics in a TimestampAssigner or
WatermarkGenerator?
Best,
Dawid
On 15/09/2020 09:44, Yunhui Han wrote:
> Hi, all
>
> I use influxdb to monitor flink metrics. However, I am confused with
> the metric taskmanager job_task_operator_KafkaConsumer_count. What
> does it mean? Is it OK for that the value is around 150 ?
> image.png
> Your Sincerely