You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (Jira)" <ji...@apache.org> on 2021/10/20 02:37:00 UTC
[jira] [Commented] (SPARK-36763) Pull out ordering expressions
[ https://issues.apache.org/jira/browse/SPARK-36763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17430847#comment-17430847 ]
Apache Spark commented on SPARK-36763:
--------------------------------------
User 'RabbidHY' has created a pull request for this issue:
https://github.com/apache/spark/pull/34334
> Pull out ordering expressions
> -----------------------------
>
> Key: SPARK-36763
> URL: https://issues.apache.org/jira/browse/SPARK-36763
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.3.0
> Reporter: Yuming Wang
> Priority: Major
>
> Similar to [PullOutGroupingExpressions|https://github.com/apache/spark/blob/7fd3f8f9ec55b364525407213ba1c631705686c5/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/PullOutGroupingExpressions.scala#L48]. We can pull out ordering expressions to improve order performance. For example:
> {code:scala}
> sql("create table t1(a int, b int) using parquet")
> sql("insert into t1 values (1, 2)")
> sql("insert into t1 values (3, 4)")
> sql("select * from t1 order by a - b").explain
> {code}
> {noformat}
> == Physical Plan ==
> AdaptiveSparkPlan isFinalPlan=false
> +- Sort [(a#12 - b#13) ASC NULLS FIRST], true, 0
> +- Exchange rangepartitioning((a#12 - b#13) ASC NULLS FIRST, 5), ENSURE_REQUIREMENTS, [id=#39]
> +- FileScan parquet default.t1[a#12,b#13]
> {noformat}
> The {{Subtract}} will be evaluated 4 times.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org