You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Andrew Otto <ot...@wikimedia.org> on 2023/02/06 19:48:24 UTC
Kafka Sink Kafka Producer metrics?
Hi!
Kafka Source will emit KafkaConsumer metrics
<https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/kafka/#kafka-consumer-metrics>
.
It looks like Kafka Sink
<https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/kafka/#monitoring-1>
does not emit KafkaProducer metrics
<https://kafka.apache.org/documentation/#producer_monitoring>. Is this
correct? If so, why not?
Thanks,
-Andrew Otto
Wikimedia Foundation
Re: Kafka Sink Kafka Producer metrics?
Posted by Andrew Otto <ot...@wikimedia.org>.
Wow, not sure how I missed that. Thank you.
On Mon, Feb 6, 2023 at 9:22 PM Mason Chen <ma...@gmail.com> wrote:
> Hi Andrew,
>
> I misread the docs: `register.producer.metrics` is mentioned here, but it
> is not on by default.
> https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/kafka/#kafka-connector-metrics
>
> Best,
> Mason
>
> On Mon, Feb 6, 2023 at 6:19 PM Mason Chen <ma...@gmail.com> wrote:
>
>> Hi Andrew,
>>
>> Unfortunately, the functionality is undocumented, but you can set the
>> property `register.producer.metrics` to true in your Kafka client
>> properties map. This is a JIRA to document the feature:
>> https://issues.apache.org/jira/browse/FLINK-30932
>>
>> Best,
>> Mason
>>
>> On Mon, Feb 6, 2023 at 11:49 AM Andrew Otto <ot...@wikimedia.org> wrote:
>>
>>> Hi!
>>>
>>> Kafka Source will emit KafkaConsumer metrics
>>> <https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/kafka/#kafka-consumer-metrics>
>>> .
>>>
>>> It looks like Kafka Sink
>>> <https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/kafka/#monitoring-1>
>>> does not emit KafkaProducer metrics
>>> <https://kafka.apache.org/documentation/#producer_monitoring>. Is this
>>> correct? If so, why not?
>>>
>>> Thanks,
>>> -Andrew Otto
>>> Wikimedia Foundation
>>>
>>
Re: Kafka Sink Kafka Producer metrics?
Posted by Mason Chen <ma...@gmail.com>.
Hi Andrew,
I misread the docs: `register.producer.metrics` is mentioned here, but it
is not on by default.
https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/kafka/#kafka-connector-metrics
Best,
Mason
On Mon, Feb 6, 2023 at 6:19 PM Mason Chen <ma...@gmail.com> wrote:
> Hi Andrew,
>
> Unfortunately, the functionality is undocumented, but you can set the
> property `register.producer.metrics` to true in your Kafka client
> properties map. This is a JIRA to document the feature:
> https://issues.apache.org/jira/browse/FLINK-30932
>
> Best,
> Mason
>
> On Mon, Feb 6, 2023 at 11:49 AM Andrew Otto <ot...@wikimedia.org> wrote:
>
>> Hi!
>>
>> Kafka Source will emit KafkaConsumer metrics
>> <https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/kafka/#kafka-consumer-metrics>
>> .
>>
>> It looks like Kafka Sink
>> <https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/kafka/#monitoring-1>
>> does not emit KafkaProducer metrics
>> <https://kafka.apache.org/documentation/#producer_monitoring>. Is this
>> correct? If so, why not?
>>
>> Thanks,
>> -Andrew Otto
>> Wikimedia Foundation
>>
>
Re: Kafka Sink Kafka Producer metrics?
Posted by Mason Chen <ma...@gmail.com>.
Hi Andrew,
Unfortunately, the functionality is undocumented, but you can set the
property `register.producer.metrics` to true in your Kafka client
properties map. This is a JIRA to document the feature:
https://issues.apache.org/jira/browse/FLINK-30932
Best,
Mason
On Mon, Feb 6, 2023 at 11:49 AM Andrew Otto <ot...@wikimedia.org> wrote:
> Hi!
>
> Kafka Source will emit KafkaConsumer metrics
> <https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/kafka/#kafka-consumer-metrics>
> .
>
> It looks like Kafka Sink
> <https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/kafka/#monitoring-1>
> does not emit KafkaProducer metrics
> <https://kafka.apache.org/documentation/#producer_monitoring>. Is this
> correct? If so, why not?
>
> Thanks,
> -Andrew Otto
> Wikimedia Foundation
>