You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Cheng Su (Jira)" <ji...@apache.org> on 2021/03/03 06:49:00 UTC
[jira] [Commented] (SPARK-34602) Improve PlanStabilitySuite to
output # of operators information
[ https://issues.apache.org/jira/browse/SPARK-34602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17294321#comment-17294321 ]
Cheng Su commented on SPARK-34602:
----------------------------------
Will submit a PR shortly.
> Improve PlanStabilitySuite to output # of operators information
> ---------------------------------------------------------------
>
> Key: SPARK-34602
> URL: https://issues.apache.org/jira/browse/SPARK-34602
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.2.0
> Reporter: Cheng Su
> Priority: Minor
>
> Based on discussion from [https://github.com/apache/spark/pull/31708#issuecomment-789444406,] plan check is really good to catch plan difference now per PR. However, it's quite hard to debug given the physical query plan can be very long and verbose for complicated query. It would be good if we can print out the information for important operators, e.g. # of shuffles, sub-queries, reused shuffle, join, aggregation, etc.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org