[jira] [Resolved] (SPARK-21414) Buffer in SlidingWindowFunctionFrame could be big though window is small

["Wenchen Fan (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-21414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wenchen Fan resolved SPARK-21414.
---------------------------------
       Resolution: Fixed
    Fix Version/s: 2.3.0
                   2.2.1

Issue resolved by pull request 18634
[https://github.com/apache/spark/pull/18634]

> Buffer in SlidingWindowFunctionFrame could be big though window is small
> ------------------------------------------------------------------------
>
>                 Key: SPARK-21414
>                 URL: https://issues.apache.org/jira/browse/SPARK-21414
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.1.1
>            Reporter: jin xing
>             Fix For: 2.2.1, 2.3.0
>
>
> In {{SlidingWindowFunctionFrame}}, it is now adding all rows to the buffer for which the input row value is equal to or less than the output row upper bound, then drop all rows from the buffer for which the input row value is smaller than the output row lower bound.
> This could result in the buffer is very big though the window is small. 
> For example: 
> {{select a, b, sum(a) over (partition by b order by a range between 1000000 following and 1000001 following) from table}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org