[jira] [Created] (FLINK-40075) Optimize WELFORD_M2 aggregate accumulator by using RowData instead of STRUCTURED POJO accumulator

["dylanhz (Jira)" <ji...@apache.org>]

dylanhz created FLINK-40075:
-------------------------------

             Summary: Optimize WELFORD_M2 aggregate accumulator by using RowData instead of STRUCTURED POJO accumulator
                 Key: FLINK-40075
                 URL: https://issues.apache.org/jira/browse/FLINK-40075
             Project: Flink
          Issue Type: Improvement
          Components: Table SQL / Runtime
            Reporter: dylanhz


{{WelfordM2AggFunction}} is used for variance-related aggregates such as {{{}VAR_POP{}}}, {{{}VAR_SAMP{}}}, {{{}STDDEV_POP{}}}, and {{{}STDDEV_SAMP{}}}.

Currently its accumulator is a structured POJO type. Using {{RowData}} as the accumulator could avoid unnecessary structured object conversion in generated aggregate code and improve performance on hot aggregate paths.

This should be investigated as a separate optimization because changing the accumulator from {{STRUCTURED<WelfordM2Accumulator>}} to {{ROW<n, mean, m2>}} may affect compiled plan and streaming state/savepoint compatibility.

*Validation*
The implementation should include:
 * variance/stddev aggregate tests,
 * compiled plan restore coverage,
 * state/savepoint or serializer compatibility checks,
 * performance evidence for the reduced conversion overhead.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)