[Feature Proposal] Expose Kafka Client Metrics in Beam

[Osvaldo Fernandez <oz...@gmail.com>]

All,

I've shared with you a link below to a document detailing a limitation of
Beam's metric system, and its impact on KafkaIO and Kafka clients as a
whole. I have outlined a handful of proposed solutions, and I'd greatly
appreciate your time looking at this. Thank you all in advance!

Best,
Ozzie Fernandez

https://docs.google.com/document/d/19JyO-5NnWJMVtnPq0T1ZIo9T2AWpA1rZsYuIVMlCVyM/edit?usp=sharing

Re: [Feature Proposal] Expose Kafka Client Metrics in Beam

[Danny McCormick via dev <de...@beam.apache.org>]

Thanks for putting this together! I am generally in favor of a DIY
PTransform option for now to get the value of these metrics in the shorter
term. I am interested in the alternative of restructuring metrics, but I
agree that we shouldn't block all progress on this.

Thanks,
Danny

On Wed, Feb 26, 2025 at 1:20 PM Osvaldo Fernandez <oz...@gmail.com>
wrote:

> All,
>
> I've shared with you a link below to a document detailing a limitation of
> Beam's metric system, and its impact on KafkaIO and Kafka clients as a
> whole. I have outlined a handful of proposed solutions, and I'd greatly
> appreciate your time looking at this. Thank you all in advance!
>
> Best,
> Ozzie Fernandez
>
>
> https://docs.google.com/document/d/19JyO-5NnWJMVtnPq0T1ZIo9T2AWpA1rZsYuIVMlCVyM/edit?usp=sharing
>