You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Dongjoon Hyun (Jira)" <ji...@apache.org> on 2025/10/24 22:17:00 UTC

[jira] [Resolved] (SPARK-53885) Frequency estimation functions

     [ https://issues.apache.org/jira/browse/SPARK-53885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dongjoon Hyun resolved SPARK-53885.
-----------------------------------
    Fix Version/s: 4.1.0
       Resolution: Fixed

> Frequency estimation functions
> ------------------------------
>
>                 Key: SPARK-53885
>                 URL: https://issues.apache.org/jira/browse/SPARK-53885
>             Project: Spark
>          Issue Type: New Feature
>          Components: SQL
>    Affects Versions: 4.1.0
>            Reporter: Gengliang Wang
>            Assignee: Yuchuan Huang
>            Priority: Major
>             Fix For: 4.1.0
>
>
> Introduce the following frequency estimation functions:
>  * *APPROX_TOP_K(expr[, k[, maxItemsTracked]])* – Returns an approximate list of the top _k_ most frequent values in the input expression using a probabilistic algorithm.
>  * *APPROX_TOP_K_ACCUMULATE(expr[, maxItemsTracked])* – Creates a state object that accumulates frequency statistics for use in later estimation or combination.
>  * *APPROX_TOP_K_ESTIMATE(state [, k] ])* – Extracts and returns the approximate top _k_ values and their estimated frequencies from an accumulated state.
>  * *APPROX_TOP_K_COMBINE(expr[, maxItemsTracked])* – Merges intermediate APPROX_TOP_K state objects, allowing distributed or parallel computation.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org