[jira] [Commented] (SPARK-38578) Avoid unnecessary sort in FileFormatWriter if user has specified sort in AQE

["Apache Spark (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-38578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17509848#comment-17509848 ] 

Apache Spark commented on SPARK-38578:
--------------------------------------

User 'ulysses-you' has created a pull request for this issue:
https://github.com/apache/spark/pull/35924

> Avoid unnecessary sort in FileFormatWriter if user has specified sort in AQE
> ----------------------------------------------------------------------------
>
>                 Key: SPARK-38578
>                 URL: https://issues.apache.org/jira/browse/SPARK-38578
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>    Affects Versions: 3.3.0
>            Reporter: XiDuo You
>            Priority: Major
>
> FileFormatWriter will check and add an implicit sort for dynamic partition columns or bucket columns according to the input physical plan. The check became always failure since AQE AdaptiveSparkPlanExec has no outputOrdering.
> That casues a redundant sort if user has specified a sort which satisfies the required ordering (dynamic partition and bucket columns).



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org