[jira] [Commented] (SPARK-45649) Unify the prepare framework for `OffsetWindowFunctionFrame`

["Wenchen Fan (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-45649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17788306#comment-17788306 ] 

Wenchen Fan commented on SPARK-45649:
-------------------------------------

This has a bug. I've uploaded the test data. You can reproduce the bug via
{code:java}
spark.read.parquet("/tmp/test_table.parquet").createOrReplaceTempView("test_table")
spark.sql("SELECT nullable_struct_col, id, lag(long_mixedparts, 5) OVER (PARTITION BY nullable_struct_col ORDER BY id ) FROM test_table").collect {code}

> Unify the prepare framework for `OffsetWindowFunctionFrame`
> -----------------------------------------------------------
>
>                 Key: SPARK-45649
>                 URL: https://issues.apache.org/jira/browse/SPARK-45649
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 4.0.0
>            Reporter: Jiaan Geng
>            Assignee: Jiaan Geng
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 4.0.0
>
>         Attachments: test_table.parquet.zip
>
>
> Currently, the implementation the `prepare` of  all the `OffsetWindowFunctionFrame` have the same code logic show below.
> ```
>   override def prepare(rows: ExternalAppendOnlyUnsafeRowArray): Unit = {
>     if (offset > rows.length) {
>       fillDefaultValue(EmptyRow)
>     } else {
>       resetStates(rows)
>       if (ignoreNulls) {
>         ...
>       } else {
>         ...
>       }
>     }
>   }
> ```



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org