[jira] [Updated] (FLINK-29717) Supports hive udaf such as sum/count by native implementation

["Xintong Song (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-29717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Xintong Song updated FLINK-29717:
---------------------------------
    Fix Version/s: 1.18.0
                       (was: 1.17.0)

> Supports hive udaf  such as sum/count by native implementation 
> ---------------------------------------------------------------
>
>                 Key: FLINK-29717
>                 URL: https://issues.apache.org/jira/browse/FLINK-29717
>             Project: Flink
>          Issue Type: Improvement
>          Components: Connectors / Hive, Table SQL / Runtime
>            Reporter: dalongliu
>            Priority: Major
>             Fix For: 1.18.0
>
>
> The current Flink side of the Hive UDAF has a unified encapsulation HiveGenericUDAF, and the intermediate result type of the aggregation function is encapsulated as RAW type, which is a variable-length data type that is serialized and deserialized by default using the Kryo serializer, so BinaryRowData does not support in-place updates to this type, which also leads to aggregation function that uses the RAW type as an intermediate aggregation buffer and cannot use hash-based aggregation strategies. Since the intermediate state type of Hive's UDAF is RAW type, it also cannot use hash-based aggregation strategy, and the overall performance in TPC-DS scenario is more than 2 times than the performance of Flink's built-in function. After some research, here we propose implementing this common udaf in a native way.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)