[jira] [Commented] (FLINK-40293) Add UDF metric config options and instrument sync scalar and table UDF calls

["Peter Huang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-40293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18122158#comment-18122158 ] 

Peter Huang commented on FLINK-40293:
-------------------------------------

Merged to master as 44db1fc214fe05324370217d7cb7b5ca046b7bbf

> Add UDF metric config options and instrument sync scalar and table UDF calls
> ----------------------------------------------------------------------------
>
>                 Key: FLINK-40293
>                 URL: https://issues.apache.org/jira/browse/FLINK-40293
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Table SQL / API, Table SQL / Planner
>            Reporter: Weiqing Yang
>            Assignee: Weiqing Yang
>            Priority: Major
>              Labels: pull-request-available
>
> This is the second of four sub-tasks implementing FLIP-485.
> FLIP: https://cwiki.apache.org/confluence/spaces/FLINK/pages/373885706/FLIP-485+Add+UDF+Metrics
> Vote thread: https://lists.apache.org/thread/symqpswsohl2s5wmtkcw0jjp1w5dot0n
> Add the two configuration options and the first end-to-end slice of the feature: synchronous scalar and table UDF calls instrumented at code generation.
> Configuration, both {{@PublicEvolving}} additions to ExecutionConfigOptions:
> * {{table.exec.udf-metric-enabled}}, default false, enables the feature.
> * {{table.exec.udf-metric.sample-interval}}, default 100, measures every Nth invocation.
> Instrumentation:
> * Wrap the generated {{eval}} call site for synchronous scalar and table UDFs in BridgingFunctionGenUtil with sampled timing and exception counting via the UdfMetrics helper.
> * Cache one handle per (operator, udf name) in CodeGeneratorContext, so repeated call sites of the same function within one operator share a handle. Without sharing, the second registration is dropped and only one call site's timings survive.
> * Meter only user functions on the modern BridgingSqlFunction stack, through an opt-in name passed from BridgingSqlFunctionCallGen. Lookup-join, ML-predict, vector-search, legacy CallGens, and PROCESS_TABLE functions are not metered.
> The instrumentation is emitted at code generation only when the feature is enabled, so the generated operator is byte-identical to today when it is disabled. When enabled, the non-sampled path costs a single integer increment. Exceptions are counted on every invocation and are not sampled.
> Covered by a new UdfMetricsITCase using InMemoryReporter: sync scalar and table functions, metric naming and scope, exception counting, the enabled/disabled gate, a lookup join not being metered, handle sharing across repeated call sites versus separate handles for distinct functions, and udfProcessingTime reflecting a real induced delay.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)