You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/02/09 13:02:02 UTC

[GitHub] [spark] Eric5553 opened a new pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Eric5553 opened a new pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509
 
 
   ### What changes were proposed in this pull request?
   The style of **EXPLAIN FORMATTED** output needs to be improved. We’ve already got some observations/ideas in
   https://github.com/apache/spark/pull/27368#discussion_r376694496. 
   
   Observations/Ideas:
   1.Using comma as the separator is not clear, especially commas are used inside the expressions too.
   2.Show the column counts first? For example, `Results [4]: …`
   3.Currently the attribute names are automatically generated, this need to refined.
   4.Add arguments field in common implementations as **EXPLAIN EXTENDED** did by calling `argString` in `TreeNode.simpleString`. This will eliminate most existing minor differences between 
   **EXPLAIN EXTENDED** and **EXPLAIN FORMATTED**.
   
   This PR is currently addressing comments 2 & 4, and open for more discussions on improving readability.
   
   
   ### Why are the changes needed?
   The readability of EXPLAIN FORMATTED need to be improved, which will help user better understand the query plan.
   
   
   ### Does this PR introduce any user-facing change?
   Yes, EXPLAIN FORMATTED output style changed.
   
   
   ### How was this patch tested?
   Update expect results of test cases in explain.sql 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586541746
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23215/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-584105643
 
 
   Merged build finished. Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] maryannxue commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
maryannxue commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r378408175
 
 

 ##########
 File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala
 ##########
 @@ -58,9 +58,9 @@ case class SortMergeJoinExec(
     } else "None"
     s"""
        |(${ExplainUtils.getOpId(this)}) $nodeName ${ExplainUtils.getCodegenId(this)}
-       |Left keys : ${leftKeys}
-       |Right keys: ${rightKeys}
-       |Join condition : ${joinCondStr}
+       |${ExplainUtils.generateFieldString("Left keys", leftKeys)}
 
 Review comment:
   It might not be related to this PR, but can we do the same thing as https://github.com/apache/spark/pull/27368/files#diff-ddb517fe44ae649ddda3c733c2adcb76R70 for joins? Just for symmetry and future handiness.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Eric5553 commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
Eric5553 commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583846013
 
 
   cc @gatorsmile @cloud-fan @maropu @dilipbiswal 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589547250
 
 
   retest this please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583906932
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22867/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586746962
 
 
   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118507/
   Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586555899
 
 
   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118458/
   Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r381960043
 
 

 ##########
 File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala
 ##########
 @@ -189,9 +189,16 @@ abstract class QueryPlan[PlanType <: QueryPlan[PlanType]] extends TreeNode[PlanT
     val codegenIdStr =
       getTagValue(QueryPlan.CODEGEN_ID_TAG).map(id => s"[codegen id : $id]").getOrElse("")
     val operatorId = getTagValue(QueryPlan.OP_ID_TAG).map(id => s"$id").getOrElse("unknown")
-    s"""
+    val argumentString = argString(SQLConf.get.maxToStringFields)
+
+    val result = s"""
        |($operatorId) $nodeName $codegenIdStr
      """.stripMargin
+    if (argumentString != null && !argumentString.isEmpty) {
+      s"""${result} |Arguments: $argumentString\n""".stripMargin
 
 Review comment:
   This is too hard to read. How about
   ```
   val result = s"($operatorId) $nodeName $codegenIdStr\n"
   if (argumentString.nonEmpty) {
     result + s"Arguments: $argumentString\n"
   } else {
     result
   }
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586635856
 
 
   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118480/
   Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] maropu commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
maropu commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583906728
 
 
   ok to test

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589013652
 
 
   Let's open a new PR for that.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Eric5553 commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
Eric5553 commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r379549006
 
 

 ##########
 File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala
 ##########
 @@ -58,9 +58,9 @@ case class SortMergeJoinExec(
     } else "None"
     s"""
        |(${ExplainUtils.getOpId(this)}) $nodeName ${ExplainUtils.getCodegenId(this)}
-       |Left keys : ${leftKeys}
-       |Right keys: ${rightKeys}
-       |Join condition : ${joinCondStr}
+       |${ExplainUtils.generateFieldString("Left keys", leftKeys)}
 
 Review comment:
   Make `HashJoin` extend `BinaryExecNode`, and `ShuffledHashJoinExec/BroadcastHashJoinExec` extend `HashJoin`, right? Yea, I can make it here together :-)

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586724474
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586541744
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583945158
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22875/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586635856
 
 
   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118480/
   Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586383646
 
 
   **[Test build #118441 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118441/testReport)** for PR 27509 at commit [`9d52f92`](https://github.com/apache/spark/commit/9d52f92c47adb4075c2e53956480e844db3de12b).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586555761
 
 
   **[Test build #118458 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118458/testReport)** for PR 27509 at commit [`9d52f92`](https://github.com/apache/spark/commit/9d52f92c47adb4075c2e53956480e844db3de12b).
    * This patch **fails PySpark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589544096
 
 
   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118745/
   Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589548269
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586541393
 
 
   **[Test build #118458 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118458/testReport)** for PR 27509 at commit [`9d52f92`](https://github.com/apache/spark/commit/9d52f92c47adb4075c2e53956480e844db3de12b).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Eric5553 commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
Eric5553 commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r382036144
 
 

 ##########
 File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala
 ##########
 @@ -189,9 +189,16 @@ abstract class QueryPlan[PlanType <: QueryPlan[PlanType]] extends TreeNode[PlanT
     val codegenIdStr =
       getTagValue(QueryPlan.CODEGEN_ID_TAG).map(id => s"[codegen id : $id]").getOrElse("")
     val operatorId = getTagValue(QueryPlan.OP_ID_TAG).map(id => s"$id").getOrElse("unknown")
-    s"""
+    val argumentString = argString(SQLConf.get.maxToStringFields)
+
+    val result = s"""
        |($operatorId) $nodeName $codegenIdStr
      """.stripMargin
+    if (argumentString != null && !argumentString.isEmpty) {
+      s"""${result} |Arguments: $argumentString\n""".stripMargin
 
 Review comment:
   It seems trim will break the first line padding of result.
   
   `result.trim + s"\nArguments: $argumentString\n"`
   Shows
   ```
   (3) ObjectHashAggregate 
   Input [2]: [key#x, val#x]
   Keys [1]: [key#x]
   Functions [1]: [partial_collect_set(val#x, 0, 0)]
   Aggregate Attributes [1]: [buf#x]
   Results [2]: [key#x, buf#x]
        (4) Exchange 
   Input [2]: [key#x, buf#x]
   Arguments: hashpartitioning(key#x, 4), true, [id=#x]
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Eric5553 commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
Eric5553 commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r382003862
 
 

 ##########
 File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala
 ##########
 @@ -189,9 +189,16 @@ abstract class QueryPlan[PlanType <: QueryPlan[PlanType]] extends TreeNode[PlanT
     val codegenIdStr =
       getTagValue(QueryPlan.CODEGEN_ID_TAG).map(id => s"[codegen id : $id]").getOrElse("")
     val operatorId = getTagValue(QueryPlan.OP_ID_TAG).map(id => s"$id").getOrElse("unknown")
-    s"""
+    val argumentString = argString(SQLConf.get.maxToStringFields)
+
+    val result = s"""
        |($operatorId) $nodeName $codegenIdStr
      """.stripMargin
+    if (argumentString != null && !argumentString.isEmpty) {
+      s"""${result} |Arguments: $argumentString\n""".stripMargin
 
 Review comment:
   I see, thanks!

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586724474
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583935858
 
 
   Merged build finished. Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-584004522
 
 
   **[Test build #118138 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118138/testReport)** for PR 27509 at commit [`4d0fcff`](https://github.com/apache/spark/commit/4d0fcffc6faf5439607379d013ebf2bb8ba3a3ab).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586541393
 
 
   **[Test build #118458 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118458/testReport)** for PR 27509 at commit [`9d52f92`](https://github.com/apache/spark/commit/9d52f92c47adb4075c2e53956480e844db3de12b).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589545727
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583906930
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589544089
 
 
   Merged build finished. Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Eric5553 commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
Eric5553 commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r379840480
 
 

 ##########
 File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala
 ##########
 @@ -58,9 +58,9 @@ case class SortMergeJoinExec(
     } else "None"
     s"""
        |(${ExplainUtils.getOpId(this)}) $nodeName ${ExplainUtils.getCodegenId(this)}
-       |Left keys : ${leftKeys}
-       |Right keys: ${rightKeys}
-       |Join condition : ${joinCondStr}
+       |${ExplainUtils.generateFieldString("Left keys", leftKeys)}
 
 Review comment:
   https://github.com/apache/spark/pull/27595 opened for this follow-up.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-584001160
 
 
   Merged build finished. Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586746962
 
 
   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118507/
   Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r381962211
 
 

 ##########
 File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlan.scala
 ##########
 @@ -531,10 +538,16 @@ trait UnaryExecNode extends SparkPlan {
 
   override final def children: Seq[SparkPlan] = child :: Nil
   override def verboseStringWithOperatorId(): String = {
-    s"""
-       |(${ExplainUtils.getOpId(this)}) $nodeName ${ExplainUtils.getCodegenId(this)}
-       |Input: ${child.output.mkString("[", ", ", "]")}
-     """.stripMargin
+    val argumentString = argString(SQLConf.get.maxToStringFields)
+    val result = s"""
+         |(${ExplainUtils.getOpId(this)}) $nodeName ${ExplainUtils.getCodegenId(this)}
+         |${ExplainUtils.generateFieldString("Input", child.output)}
+       """.stripMargin
+    if (argumentString != null && !argumentString.isEmpty) {
+      s"""${result} |Arguments: $argumentString\n""".stripMargin
 
 Review comment:
   let's not use multiline string if it's not multiline

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589544096
 
 
   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118745/
   Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] maropu commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
maropu commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583906771
 
 
   Thanks for your work, @Eric5553 !

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-584105653
 
 
   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118138/
   Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Eric5553 commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
Eric5553 commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-584002561
 
 
   retest this please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan closed pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583906932
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22867/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Eric5553 commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
Eric5553 commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r382062369
 
 

 ##########
 File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala
 ##########
 @@ -189,9 +189,16 @@ abstract class QueryPlan[PlanType <: QueryPlan[PlanType]] extends TreeNode[PlanT
     val codegenIdStr =
       getTagValue(QueryPlan.CODEGEN_ID_TAG).map(id => s"[codegen id : $id]").getOrElse("")
     val operatorId = getTagValue(QueryPlan.OP_ID_TAG).map(id => s"$id").getOrElse("unknown")
-    s"""
+    val argumentString = argString(SQLConf.get.maxToStringFields)
+
+    val result = s"""
        |($operatorId) $nodeName $codegenIdStr
      """.stripMargin
+    if (argumentString != null && !argumentString.isEmpty) {
+      s"""${result} |Arguments: $argumentString\n""".stripMargin
 
 Review comment:
   Yea, it works with better code style. Thanks a lot!

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583945157
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583935627
 
 
   **[Test build #118102 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118102/testReport)** for PR 27509 at commit [`9d30fad`](https://github.com/apache/spark/commit/9d30fada81864fa1c086f7ca3285ed3b86253fd0).
    * This patch **fails PySpark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Eric5553 commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
Eric5553 commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586386452
 
 
   Also trying to address the improvement https://github.com/apache/spark/pull/27368#discussion_r376927143 here in this PR. Add new string format for `Expression` to avoid much footprint on existing behaviors. There are still some `#exprId` not removed, I'm keep working on this.
   
   @cloud-fan
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r381961568
 
 

 ##########
 File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlan.scala
 ##########
 @@ -512,10 +513,16 @@ trait LeafExecNode extends SparkPlan {
   override final def children: Seq[SparkPlan] = Nil
   override def producedAttributes: AttributeSet = outputSet
   override def verboseStringWithOperatorId(): String = {
-    s"""
-       |(${ExplainUtils.getOpId(this)}) $nodeName ${ExplainUtils.getCodegenId(this)}
-       |Output: ${producedAttributes.mkString("[", ", ", "]")}
-     """.stripMargin
+    val argumentString = argString(SQLConf.get.maxToStringFields)
+    val result = s"""
+         |(${ExplainUtils.getOpId(this)}) $nodeName ${ExplainUtils.getCodegenId(this)}
+         |${ExplainUtils.generateFieldString("Output", producedAttributes)}
+       """.stripMargin
+    if (argumentString != null && !argumentString.isEmpty) {
+      s"""${result} |Arguments: $argumentString\n""".stripMargin
 
 Review comment:
   same as https://github.com/apache/spark/pull/27509/files#r381960043

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589478104
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Eric5553 commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
Eric5553 commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586540122
 
 
   retest this please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589548279
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23517/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583843498
 
 
   Can one of the admins verify this patch?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Eric5553 commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
Eric5553 commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r379549380
 
 

 ##########
 File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala
 ##########
 @@ -189,8 +189,11 @@ abstract class QueryPlan[PlanType <: QueryPlan[PlanType]] extends TreeNode[PlanT
     val codegenIdStr =
       getTagValue(QueryPlan.CODEGEN_ID_TAG).map(id => s"[codegen id : $id]").getOrElse("")
     val operatorId = getTagValue(QueryPlan.OP_ID_TAG).map(id => s"$id").getOrElse("unknown")
+    val argStr = argString(SQLConf.get.maxToStringFields)
+
     s"""
        |($operatorId) $nodeName $codegenIdStr
+       |Arguments: ${if (argStr != null && !argStr.isEmpty) argStr else "None"}
 
 Review comment:
   +1,  thanks for the suggestion!

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-584105643
 
 
   Merged build finished. Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] maryannxue commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
maryannxue commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r378409214
 
 

 ##########
 File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala
 ##########
 @@ -189,8 +189,11 @@ abstract class QueryPlan[PlanType <: QueryPlan[PlanType]] extends TreeNode[PlanT
     val codegenIdStr =
       getTagValue(QueryPlan.CODEGEN_ID_TAG).map(id => s"[codegen id : $id]").getOrElse("")
     val operatorId = getTagValue(QueryPlan.OP_ID_TAG).map(id => s"$id").getOrElse("unknown")
+    val argStr = argString(SQLConf.get.maxToStringFields)
+
     s"""
        |($operatorId) $nodeName $codegenIdStr
+       |Arguments: ${if (argStr != null && !argStr.isEmpty) argStr else "None"}
 
 Review comment:
   Do we need to mute "Arguments" if no arguments instead of printing "None"?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586614521
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589548269
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r381960415
 
 

 ##########
 File path: sql/core/src/main/scala/org/apache/spark/sql/execution/ExplainUtils.scala
 ##########
 @@ -182,6 +181,17 @@ object ExplainUtils {
     }
   }
 
+  /**
+   * Generate detailed field string with different format based on type of input value
+   */
+  def generateFieldString(fieldName: String, values: Any): String = values match {
+    case iter: Iterable[_] if (iter.size == 0) => s"${fieldName}: []"
 
 Review comment:
   nit: `if iter.size == 0`

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583935858
 
 
   Merged build finished. Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Eric5553 commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
Eric5553 commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589545376
 
 
   retest this please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-584001171
 
 
   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118110/
   Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r381960043
 
 

 ##########
 File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala
 ##########
 @@ -189,9 +189,16 @@ abstract class QueryPlan[PlanType <: QueryPlan[PlanType]] extends TreeNode[PlanT
     val codegenIdStr =
       getTagValue(QueryPlan.CODEGEN_ID_TAG).map(id => s"[codegen id : $id]").getOrElse("")
     val operatorId = getTagValue(QueryPlan.OP_ID_TAG).map(id => s"$id").getOrElse("unknown")
-    s"""
+    val argumentString = argString(SQLConf.get.maxToStringFields)
+
+    val result = s"""
        |($operatorId) $nodeName $codegenIdStr
      """.stripMargin
+    if (argumentString != null && !argumentString.isEmpty) {
+      s"""${result} |Arguments: $argumentString\n""".stripMargin
 
 Review comment:
   This is too hard to read. How about
   ```
   if (argumentString.nonEmpty) {
     result + s"Arguments: $argumentString\n"
   } else {
     result
   }
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586541744
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-584000759
 
 
   **[Test build #118110 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118110/testReport)** for PR 27509 at commit [`4d0fcff`](https://github.com/apache/spark/commit/4d0fcffc6faf5439607379d013ebf2bb8ba3a3ab).
    * This patch **fails due to an unknown error code, -9**.
    * This patch merges cleanly.
    * This patch adds no public classes.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Eric5553 edited a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
Eric5553 edited a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589012151
 
 
   @cloud-fan Sorry for missed the failed unit test(just found where to get the jenkins error log...). I'll fix the pyspark test and address comments today. 
   
   And I got an implementation of removing useless `#{exprId.id}`, but it needs a lot of changes for existing tests. Should I commit here in this PR or open a separate one? Thanks

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589673058
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118765/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589478104
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589548279
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23517/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589673058
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118765/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583935860
 
 
   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118102/
   Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586541746
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23215/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589545735
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23515/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-584005086
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22904/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r381961389
 
 

 ##########
 File path: sql/core/src/main/scala/org/apache/spark/sql/execution/ExplainUtils.scala
 ##########
 @@ -182,6 +181,17 @@ object ExplainUtils {
     }
   }
 
+  /**
+   * Generate detailed field string with different format based on type of input value
+   */
+  def generateFieldString(fieldName: String, values: Any): String = values match {
+    case iter: Iterable[_] if (iter.size == 0) => s"${fieldName}: []"
+    case iter: Iterable[_] => s"${fieldName} [${iter.size}]: ${iter.mkString("[", ", ", "]")}"
+    case str: String if (str == null || str.isEmpty) => s"${fieldName}: None"
+    case str: String => s"${fieldName}: ${str}"
+    case _ => s"${fieldName}: Unknown"
 
 Review comment:
   This is not expected. Shall we just throw exception here?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586482490
 
 
   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118441/
   Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583843498
 
 
   Can one of the admins verify this patch?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r381960415
 
 

 ##########
 File path: sql/core/src/main/scala/org/apache/spark/sql/execution/ExplainUtils.scala
 ##########
 @@ -182,6 +181,17 @@ object ExplainUtils {
     }
   }
 
+  /**
+   * Generate detailed field string with different format based on type of input value
+   */
+  def generateFieldString(fieldName: String, values: Any): String = values match {
+    case iter: Iterable[_] if (iter.size == 0) => s"${fieldName}: []"
 
 Review comment:
   nit: `if iter.size == 0`

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589542104
 
 
   LGTM

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589547892
 
 
   **[Test build #118765 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118765/testReport)** for PR 27509 at commit [`1290cd5`](https://github.com/apache/spark/commit/1290cd523d6fdead5399923ea4acac450a5c2175).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Eric5553 edited a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
Eric5553 edited a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586386452
 
 
   Also trying to address the improvement https://github.com/apache/spark/pull/27368#discussion_r376927143 here in this PR. I tried with adding new format string function for `Expression`, but that caused too much code change and hard to resolve `AttributeReference.toString` which may recursively built by `name`. 
   Now I'm working on just refactor the existing `toString` functions of `Alias` and `AttributeReference`.
   
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583843597
 
 
   Can one of the admins verify this patch?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583946175
 
 
   **[Test build #118110 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118110/testReport)** for PR 27509 at commit [`4d0fcff`](https://github.com/apache/spark/commit/4d0fcffc6faf5439607379d013ebf2bb8ba3a3ab).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586379266
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586635854
 
 
   Merged build finished. Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Eric5553 commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
Eric5553 commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589012151
 
 
   @cloud-fan Sorry for missed the failed unit test(just found where to get the jenkins error log...). I'll fixed the pyspark test and address comments today. 
   
   And I got an implementation of removing useless `#{exprId.id}`, but it needs a lot of changes for existing tests. Should I commit here in this PR or open a separate one? Thanks

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586614522
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23238/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589477851
 
 
   **[Test build #118745 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118745/testReport)** for PR 27509 at commit [`1290cd5`](https://github.com/apache/spark/commit/1290cd523d6fdead5399923ea4acac450a5c2175).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Eric5553 commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
Eric5553 commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r382003758
 
 

 ##########
 File path: sql/core/src/main/scala/org/apache/spark/sql/execution/ExplainUtils.scala
 ##########
 @@ -182,6 +181,17 @@ object ExplainUtils {
     }
   }
 
+  /**
+   * Generate detailed field string with different format based on type of input value
+   */
+  def generateFieldString(fieldName: String, values: Any): String = values match {
+    case iter: Iterable[_] if (iter.size == 0) => s"${fieldName}: []"
+    case iter: Iterable[_] => s"${fieldName} [${iter.size}]: ${iter.mkString("[", ", ", "]")}"
+    case str: String if (str == null || str.isEmpty) => s"${fieldName}: None"
+    case str: String => s"${fieldName}: ${str}"
+    case _ => s"${fieldName}: Unknown"
 
 Review comment:
   Sure

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Eric5553 removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
Eric5553 removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586540122
 
 
   retest this please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] gatorsmile commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
gatorsmile commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r410718548
 
 

 ##########
 File path: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala
 ##########
 @@ -243,7 +243,7 @@ case class FilterExec(condition: Expression, child: SparkPlan)
   override def verboseStringWithOperatorId(): String = {
     s"""
        |(${ExplainUtils.getOpId(this)}) $nodeName ${ExplainUtils.getCodegenId(this)}
-       |Input     : ${child.output.mkString("[", ", ", "]")}
+       |${ExplainUtils.generateFieldString("Input", child.output)}
        |Condition : ${condition}
 
 Review comment:
   Can we remove the space before ":"?
   
   ```
   (3) Filter [codegen id : 1]
   Input [1]: [col.dots#22]
   Condition : (isnotnull(col.dots#22) AND (col.dots#22 = 500))
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586724304
 
 
   **[Test build #118507 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118507/testReport)** for PR 27509 at commit [`f0bcf12`](https://github.com/apache/spark/commit/f0bcf1280dc556447b225d6519177c694efe7da8).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586724476
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23262/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] gatorsmile commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
gatorsmile commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583951165
 
 
   Also cc @maryannxue @hvanhovell 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583945158
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22875/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586614522
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23238/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586555898
 
 
   Merged build finished. Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589673045
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586724304
 
 
   **[Test build #118507 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118507/testReport)** for PR 27509 at commit [`f0bcf12`](https://github.com/apache/spark/commit/f0bcf1280dc556447b225d6519177c694efe7da8).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-584105653
 
 
   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118138/
   Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583935860
 
 
   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118102/
   Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586555899
 
 
   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118458/
   Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-584005074
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589671937
 
 
   **[Test build #118765 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118765/testReport)** for PR 27509 at commit [`1290cd5`](https://github.com/apache/spark/commit/1290cd523d6fdead5399923ea4acac450a5c2175).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586614263
 
 
   **[Test build #118480 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118480/testReport)** for PR 27509 at commit [`9d52f92`](https://github.com/apache/spark/commit/9d52f92c47adb4075c2e53956480e844db3de12b).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586746959
 
 
   Merged build finished. Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586555898
 
 
   Merged build finished. Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586746789
 
 
   **[Test build #118507 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118507/testReport)** for PR 27509 at commit [`f0bcf12`](https://github.com/apache/spark/commit/f0bcf1280dc556447b225d6519177c694efe7da8).
    * This patch **fails PySpark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586614263
 
 
   **[Test build #118480 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118480/testReport)** for PR 27509 at commit [`9d52f92`](https://github.com/apache/spark/commit/9d52f92c47adb4075c2e53956480e844db3de12b).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586383646
 
 
   **[Test build #118441 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118441/testReport)** for PR 27509 at commit [`9d52f92`](https://github.com/apache/spark/commit/9d52f92c47adb4075c2e53956480e844db3de12b).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589545727
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583843597
 
 
   Can one of the admins verify this patch?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] maryannxue commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
maryannxue commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r379613391
 
 

 ##########
 File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala
 ##########
 @@ -58,9 +58,9 @@ case class SortMergeJoinExec(
     } else "None"
     s"""
        |(${ExplainUtils.getOpId(this)}) $nodeName ${ExplainUtils.getCodegenId(this)}
-       |Left keys : ${leftKeys}
-       |Right keys: ${rightKeys}
-       |Join condition : ${joinCondStr}
+       |${ExplainUtils.generateFieldString("Left keys", leftKeys)}
 
 Review comment:
   No. I meant creating a trait for all physical joins. It'll make pattern matching easier although we don't have this requirement right now. We could do it in a follow-up.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-584001171
 
 
   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118110/
   Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-584005074
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586634788
 
 
   **[Test build #118480 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118480/testReport)** for PR 27509 at commit [`9d52f92`](https://github.com/apache/spark/commit/9d52f92c47adb4075c2e53956480e844db3de12b).
    * This patch **fails PySpark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Eric5553 commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
Eric5553 commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r379840480
 
 

 ##########
 File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala
 ##########
 @@ -58,9 +58,9 @@ case class SortMergeJoinExec(
     } else "None"
     s"""
        |(${ExplainUtils.getOpId(this)}) $nodeName ${ExplainUtils.getCodegenId(this)}
-       |Left keys : ${leftKeys}
-       |Right keys: ${rightKeys}
-       |Join condition : ${joinCondStr}
+       |${ExplainUtils.generateFieldString("Left keys", leftKeys)}
 
 Review comment:
   PR https://github.com/apache/spark/pull/27595 opened for this follow-up.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Eric5553 commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
Eric5553 commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586613086
 
 
   retest this please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Eric5553 commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
Eric5553 commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r382036144
 
 

 ##########
 File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala
 ##########
 @@ -189,9 +189,16 @@ abstract class QueryPlan[PlanType <: QueryPlan[PlanType]] extends TreeNode[PlanT
     val codegenIdStr =
       getTagValue(QueryPlan.CODEGEN_ID_TAG).map(id => s"[codegen id : $id]").getOrElse("")
     val operatorId = getTagValue(QueryPlan.OP_ID_TAG).map(id => s"$id").getOrElse("unknown")
-    s"""
+    val argumentString = argString(SQLConf.get.maxToStringFields)
+
+    val result = s"""
        |($operatorId) $nodeName $codegenIdStr
      """.stripMargin
+    if (argumentString != null && !argumentString.isEmpty) {
+      s"""${result} |Arguments: $argumentString\n""".stripMargin
 
 Review comment:
   It seems trim will break the first line padding of result.
   
   `result.trim + s"\nArguments: $argumentString\n"`
   Shows
   ```
        (9) BroadcastExchange 
   Input [2]: [key#x, val#x]
   Arguments: HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint))), [id=#x]
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Eric5553 commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
Eric5553 commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589721555
 
 
   Thanks so much! @cloud-fan @maryannxue @maropu @gatorsmile 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-584104958
 
 
   **[Test build #118138 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118138/testReport)** for PR 27509 at commit [`4d0fcff`](https://github.com/apache/spark/commit/4d0fcffc6faf5439607379d013ebf2bb8ba3a3ab).
    * This patch **fails PySpark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589543553
 
 
   **[Test build #118745 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118745/testReport)** for PR 27509 at commit [`1290cd5`](https://github.com/apache/spark/commit/1290cd523d6fdead5399923ea4acac450a5c2175).
    * This patch **fails due to an unknown error code, -9**.
    * This patch merges cleanly.
    * This patch adds no public classes.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586379266
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-588992515
 
 
   ```
   File "/home/jenkins/workspace/SparkPullRequestBuilder/python/pyspark/sql/dataframe.py", line 282, in pyspark.sql.dataframe.DataFrame.explain
   Failed example:
       df.explain(mode="formatted")
   Differences (ndiff with -expected +actual):
         == Physical Plan ==
         * Scan ExistingRDD (1)
       + <BLANKLINE>
       + <BLANKLINE>
         (1) Scan ExistingRDD [codegen id : 1]
       - Output: [age#0, name#1]
       + Output [2]: [age#0, name#1]
       ?       ++++
       + Arguments: [age#0, name#1], MapPartitionsRDD[4] at applySchemaToPythonRDD at NativeMethodAccessorImpl.java:0, ExistingRDD, UnknownPartitioning(0)
       + <BLANKLINE>
       + <BLANKLINE>
   **********************************************************************
      1 of   3 in pyspark.sql.dataframe.DataFrame.explain
   ***Test Failed*** 1 failures.
   ```
   @Eric5553 can you fix the pyspark test? you should update `dataframe.py` `explain` method
   ```
   >>> df.explain(mode="formatted")
           == Physical Plan ==
           * Scan ExistingRDD (1)
           (1) Scan ExistingRDD [codegen id : 1]
           Output: [age#0, name#1]
   
           .. versionchanged:: 3.0.0
              Added optional argument `mode` to specify the expected output format of plans.
           """
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r382042389
 
 

 ##########
 File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala
 ##########
 @@ -189,9 +189,16 @@ abstract class QueryPlan[PlanType <: QueryPlan[PlanType]] extends TreeNode[PlanT
     val codegenIdStr =
       getTagValue(QueryPlan.CODEGEN_ID_TAG).map(id => s"[codegen id : $id]").getOrElse("")
     val operatorId = getTagValue(QueryPlan.OP_ID_TAG).map(id => s"$id").getOrElse("unknown")
-    s"""
+    val argumentString = argString(SQLConf.get.maxToStringFields)
+
+    val result = s"""
        |($operatorId) $nodeName $codegenIdStr
      """.stripMargin
+    if (argumentString != null && !argumentString.isEmpty) {
+      s"""${result} |Arguments: $argumentString\n""".stripMargin
 
 Review comment:
   how about
   ```
   val baseStr = s"($operatorId) $nodeName $codegenIdStr"
   if (argumentString.nonEmpty) {
     s"""
       |$baseStr
       |Arguments: $argumentString
     """. stripMargin
   } else {
     s"""
     |$baseStr
     """. stripMargin
   }
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Eric5553 commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
Eric5553 commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r382020326
 
 

 ##########
 File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala
 ##########
 @@ -189,9 +189,16 @@ abstract class QueryPlan[PlanType <: QueryPlan[PlanType]] extends TreeNode[PlanT
     val codegenIdStr =
       getTagValue(QueryPlan.CODEGEN_ID_TAG).map(id => s"[codegen id : $id]").getOrElse("")
     val operatorId = getTagValue(QueryPlan.OP_ID_TAG).map(id => s"$id").getOrElse("unknown")
-    s"""
+    val argumentString = argString(SQLConf.get.maxToStringFields)
+
+    val result = s"""
        |($operatorId) $nodeName $codegenIdStr
      """.stripMargin
+    if (argumentString != null && !argumentString.isEmpty) {
+      s"""${result} |Arguments: $argumentString\n""".stripMargin
 
 Review comment:
   Oh, I just remember why I tried so complicated writing here. 
   `result + s"Arguments: $argumentString\n"`  will lead to unexpected padding before Arguments. 
   Thus to avoid the padding and not using improper multiline string maybe we can use 
   `s"${result} |Arguments: $argumentString\n".stripMargin` ?
   
   Any other suggestions? Thanks @cloud-fan 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583906807
 
 
   **[Test build #118102 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118102/testReport)** for PR 27509 at commit [`9d30fad`](https://github.com/apache/spark/commit/9d30fada81864fa1c086f7ca3285ed3b86253fd0).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589544089
 
 
   Merged build finished. Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586724476
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23262/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586746959
 
 
   Merged build finished. Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586482490
 
 
   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118441/
   Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589545735
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23515/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-584004522
 
 
   **[Test build #118138 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118138/testReport)** for PR 27509 at commit [`4d0fcff`](https://github.com/apache/spark/commit/4d0fcffc6faf5439607379d013ebf2bb8ba3a3ab).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583946175
 
 
   **[Test build #118110 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118110/testReport)** for PR 27509 at commit [`4d0fcff`](https://github.com/apache/spark/commit/4d0fcffc6faf5439607379d013ebf2bb8ba3a3ab).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589477851
 
 
   **[Test build #118745 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118745/testReport)** for PR 27509 at commit [`1290cd5`](https://github.com/apache/spark/commit/1290cd523d6fdead5399923ea4acac450a5c2175).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589478109
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23496/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] gatorsmile commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
gatorsmile commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r410733762
 
 

 ##########
 File path: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala
 ##########
 @@ -76,7 +76,7 @@ trait DataSourceScanExec extends LeafExecNode {
 
     s"""
        |(${ExplainUtils.getOpId(this)}) $nodeName ${ExplainUtils.getCodegenId(this)}
-       |Output: ${producedAttributes.mkString("[", ", ", "]")}
+       |${ExplainUtils.generateFieldString("Output", producedAttributes)}
        |${metadataStr.mkString("\n")}
 
 Review comment:
   These changes are only for DSV1. Could we make the corresponding changes when using DSV2? Open the ticket https://issues.apache.org/jira/browse/SPARK-31480
   
   Also, please check the output when the schema is very long. For example, containing 250+ columns. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586614521
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586482074
 
 
   **[Test build #118441 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118441/testReport)** for PR 27509 at commit [`9d52f92`](https://github.com/apache/spark/commit/9d52f92c47adb4075c2e53956480e844db3de12b).
    * This patch **fails PySpark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586482483
 
 
   Merged build finished. Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583906930
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Eric5553 commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
Eric5553 commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r379707770
 
 

 ##########
 File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala
 ##########
 @@ -58,9 +58,9 @@ case class SortMergeJoinExec(
     } else "None"
     s"""
        |(${ExplainUtils.getOpId(this)}) $nodeName ${ExplainUtils.getCodegenId(this)}
-       |Left keys : ${leftKeys}
-       |Right keys: ${rightKeys}
-       |Join condition : ${joinCondStr}
+       |${ExplainUtils.generateFieldString("Left keys", leftKeys)}
 
 Review comment:
   Oh, yea. I just recall the conversation, thanks for your explanation :-)
   I'll submit a follow-up PR for joins accordingly. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r382022956
 
 

 ##########
 File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala
 ##########
 @@ -189,9 +189,16 @@ abstract class QueryPlan[PlanType <: QueryPlan[PlanType]] extends TreeNode[PlanT
     val codegenIdStr =
       getTagValue(QueryPlan.CODEGEN_ID_TAG).map(id => s"[codegen id : $id]").getOrElse("")
     val operatorId = getTagValue(QueryPlan.OP_ID_TAG).map(id => s"$id").getOrElse("unknown")
-    s"""
+    val argumentString = argString(SQLConf.get.maxToStringFields)
+
+    val result = s"""
        |($operatorId) $nodeName $codegenIdStr
      """.stripMargin
+    if (argumentString != null && !argumentString.isEmpty) {
+      s"""${result} |Arguments: $argumentString\n""".stripMargin
 
 Review comment:
   how about `result.trim + s"Arguments: $argumentString\n"`?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-584005086
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22904/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-584001160
 
 
   Merged build finished. Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589478109
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23496/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Eric5553 removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
Eric5553 removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-584002561
 
 
   retest this please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583945157
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586482483
 
 
   Merged build finished. Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589547892
 
 
   **[Test build #118765 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118765/testReport)** for PR 27509 at commit [`1290cd5`](https://github.com/apache/spark/commit/1290cd523d6fdead5399923ea4acac450a5c2175).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586379271
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23198/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586635854
 
 
   Merged build finished. Test FAILed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] Eric5553 commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
Eric5553 commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r382036144
 
 

 ##########
 File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala
 ##########
 @@ -189,9 +189,16 @@ abstract class QueryPlan[PlanType <: QueryPlan[PlanType]] extends TreeNode[PlanT
     val codegenIdStr =
       getTagValue(QueryPlan.CODEGEN_ID_TAG).map(id => s"[codegen id : $id]").getOrElse("")
     val operatorId = getTagValue(QueryPlan.OP_ID_TAG).map(id => s"$id").getOrElse("unknown")
-    s"""
+    val argumentString = argString(SQLConf.get.maxToStringFields)
+
+    val result = s"""
        |($operatorId) $nodeName $codegenIdStr
      """.stripMargin
+    if (argumentString != null && !argumentString.isEmpty) {
+      s"""${result} |Arguments: $argumentString\n""".stripMargin
 
 Review comment:
   It seems trim will break the first line padding of `result`.
   
   `result.trim + s"\nArguments: $argumentString\n"`
   Shows
   ```
   (3) ObjectHashAggregate 
   Input [2]: [key#x, val#x]
   Keys [1]: [key#x]
   Functions [1]: [partial_collect_set(val#x, 0, 0)]
   Aggregate Attributes [1]: [buf#x]
   Results [2]: [key#x, buf#x]
        (4) Exchange 
   Input [2]: [key#x, buf#x]
   Arguments: hashpartitioning(key#x, 4), true, [id=#x]
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-583906807
 
 
   **[Test build #118102 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118102/testReport)** for PR 27509 at commit [`9d30fad`](https://github.com/apache/spark/commit/9d30fada81864fa1c086f7ca3285ed3b86253fd0).

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589705570
 
 
   thanks, merging to master/3.0!

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-586379271
 
 
   Test PASSed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23198/
   Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#discussion_r381961714
 
 

 ##########
 File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlan.scala
 ##########
 @@ -531,10 +538,16 @@ trait UnaryExecNode extends SparkPlan {
 
   override final def children: Seq[SparkPlan] = child :: Nil
   override def verboseStringWithOperatorId(): String = {
-    s"""
-       |(${ExplainUtils.getOpId(this)}) $nodeName ${ExplainUtils.getCodegenId(this)}
-       |Input: ${child.output.mkString("[", ", ", "]")}
-     """.stripMargin
+    val argumentString = argString(SQLConf.get.maxToStringFields)
+    val result = s"""
+         |(${ExplainUtils.getOpId(this)}) $nodeName ${ExplainUtils.getCodegenId(this)}
+         |${ExplainUtils.generateFieldString("Input", child.output)}
+       """.stripMargin
+    if (argumentString != null && !argumentString.isEmpty) {
+      s"""${result} |Arguments: $argumentString\n""".stripMargin
 
 Review comment:
   ditto

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27509: [SPARK-30764][SQL] Improve the readability of EXPLAIN FORMATTED style
URL: https://github.com/apache/spark/pull/27509#issuecomment-589673045
 
 
   Merged build finished. Test PASSed.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org