You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Daniel Tenedorio (Jira)" <ji...@apache.org> on 2025/11/03 20:14:00 UTC
[jira] [Assigned] (SPARK-54138) Enforce constant configuration parameter for Hll and Theta Sketches Aggregation
[ https://issues.apache.org/jira/browse/SPARK-54138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Tenedorio reassigned SPARK-54138:
----------------------------------------
Assignee: Christopher Boumalhab
> Enforce constant configuration parameter for Hll and Theta Sketches Aggregation
> -------------------------------------------------------------------------------
>
> Key: SPARK-54138
> URL: https://issues.apache.org/jira/browse/SPARK-54138
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.1.0
> Reporter: Christopher Boumalhab
> Assignee: Christopher Boumalhab
> Priority: Minor
> Labels: pull-request-available
>
> *Description:*
> Add validation to ensure that the sketch configuration parameter (e.g., {{{}lgConfigK/lgNomEntries{}}}) in both {{HllSketchAgg}} and {{ThetaSketchAgg}} is constant. If the configuration expression ({{{}right{}}}) is not foldable, throw a {{QueryExecutionErrors.*MustBeConstantError(prettyName)}} to prevent runtime issues caused by dynamic or non-deterministic configurations.
> *Rationale:*
> The configuration parameter defines key precision and memory characteristics of HLL and Theta sketches. It must be known at analysis time to guarantee deterministic aggregation behavior and consistent accuracy. Allowing non-constant parameters could lead to unpredictable results and incorrect aggregations.
> *Acceptance Criteria:*
> * Queries fail with a clear error if the configuration parameter ({{{}right{}}}) is not foldable.
> * Constant parameters are accepted and validated with existing checks (e.g., {{{}checkLgK{}}}).
> * Existing valid queries with constant configuration values continue to work without changes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org