[jira] [Updated] (SPARK-38075) Hive script transform with order by and limit will return fake rows

["Bruce Robbins (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-38075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bruce Robbins updated SPARK-38075:
----------------------------------
    Affects Version/s: 3.2.0

> Hive script transform with order by and limit will return fake rows
> -------------------------------------------------------------------
>
>                 Key: SPARK-38075
>                 URL: https://issues.apache.org/jira/browse/SPARK-38075
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 3.2.0, 3.3.0
>            Reporter: Bruce Robbins
>            Priority: Major
>              Labels: correctness
>
> For example:
> {noformat}
> create or replace temp view t as
> select * from values
> (1),
> (2),
> (3)
> as t(a);
> select transform(a)
> USING 'cat' AS (a int)
> FROM t order by a limit 10;
> {noformat}
> This returns:
> {noformat}
> NULL
> NULL
> NULL
> 1
> 2
> 3
> {noformat}
> Without {{order by}} and {{limit}}, the query returns:
> {noformat}
> 1
> 2
> 3
> {noformat}
> Spark script transform does not have this issue. That is, if {{spark.sql.catalogImplementation=in-memory}}, Spark does not return fake rows.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org