[Discuss] FLIP-485: Flink UDF metrics

[Linda Li <li...@gmail.com>]

Hi,

  Starting a thread for adding Flink UDF metrics.
  UDFs are widely used by Flink user, but they introduce significant
challenges when it comes to debugging and performance optimization. UDFs
often act as black boxes, making it difficult to trace issues or identify
the root causes of failures, especially when exceptions occur. The lack of
dedicated metrics to quantify critical aspects such as processing time and
exception count complicates debugging as well as performance tuning and
resource allocation
   These metrics will be disabled by default to prevent performance
downgrade and can be enabled through config.
https://docs.google.com/document/d/1V03HJg7hHHcq6npuIO0PSSTDRHEB8ckSP65qJ3YV_10/edit?usp=sharing

Thanks,
Shiyuan

Re: [Discuss] FLIP-485: Flink UDF metrics

[Weiqing Yang <ya...@gmail.com>]

I synced with Linda (lindali991010@gmail.com) offline. She will no longer
be working on this proposal, so we will take over and continue the
follow-up. I will create a new discussion thread for it.


Best,
Weiqing

On Tue, Dec 24, 2024 at 7:41 PM Linda Li <li...@gmail.com> wrote:

> Hi,
>
>   Starting a thread for adding Flink UDF metrics.
>   UDFs are widely used by Flink user, but they introduce significant
> challenges when it comes to debugging and performance optimization. UDFs
> often act as black boxes, making it difficult to trace issues or identify
> the root causes of failures, especially when exceptions occur. The lack of
> dedicated metrics to quantify critical aspects such as processing time and
> exception count complicates debugging as well as performance tuning and
> resource allocation
>    These metrics will be disabled by default to prevent performance
> downgrade and can be enabled through config.
>
> https://docs.google.com/document/d/1V03HJg7hHHcq6npuIO0PSSTDRHEB8ckSP65qJ3YV_10/edit?usp=sharing
>
> Thanks,
> Shiyuan
>