You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Uroš Bojanić (Jira)" <ji...@apache.org> on 2026/08/02 22:03:00 UTC
[jira] [Resolved] (SPARK-58503) In the presence of a generator function, Having is not pushed below the Window
[ https://issues.apache.org/jira/browse/SPARK-58503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uroš Bojanić resolved SPARK-58503.
----------------------------------
Fix Version/s: 4.3.0
Resolution: Fixed
Issue resolved by pull request 57704
[https://github.com/apache/spark/pull/57704]
> In the presence of a generator function, Having is not pushed below the Window
> ------------------------------------------------------------------------------
>
> Key: SPARK-58503
> URL: https://issues.apache.org/jira/browse/SPARK-58503
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.3.0
> Reporter: Vladimir Golubev
> Assignee: Vladimir Golubev
> Priority: Major
> Labels: correctness, pull-request-available
> Fix For: 4.3.0
>
>
> SQL (with a no-op generator):
> SELECT explode(array(a)) AS col, count(*) OVER () AS cnt, a
> FROM VALUES (1), (2), (3), (NULL) AS t(a)
> GROUP BY a
> HAVING a IS NOT NULL;
> ->
> col cnt a
> 1 4 1
> 2 4 2
> 3 4 3
> Plan:
> col: int, cnt: bigint, a: int
> Filter isnotnull(a#13438)
> +- Project [col#13450, cnt#13440L, a#13438]
> +- Generate explode(_gen_input_0#13448), false, [col#13450]
> +- Window [array(a#13438) AS _gen_input_0#13448, count(1) windowspecdefinition(specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) AS cnt#13440L, a#13438]
> +- Aggregate [a#13438], [a#13438]
> +- SubqueryAlias t
> +- LocalRelation [a#13438]
> Equivalently, if there was no no-op generator explode(array(a)):
> SELECT a AS col, count(*) OVER () AS cnt, a
> FROM VALUES (1), (2), (3), (NULL) AS t(a)
> GROUP BY a
> HAVING a IS NOT NULL;
> ->
> col cnt a
> 1 3 1
> 2 3 2
> 3 3 3
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org