You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/04/02 04:51:26 UTC
[GitHub] [spark] maropu opened a new pull request #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
maropu opened a new pull request #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://spark.apache.org/contributing.html
2. Ensure you have added or run the appropriate tests for your PR: https://spark.apache.org/developer-tools.html
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][SPARK-XXXX] Your PR title ...'.
4. Be sure to keep the PR description updated to reflect all changes.
5. Please write your PR title to summarize what this PR proposes.
6. If possible, provide a concise example to reproduce the issue for a faster review.
7. If you want to add a new configuration, please read the guideline first for naming configurations in
'core/src/main/scala/org/apache/spark/internal/config/ConfigEntry.scala'.
-->
### What changes were proposed in this pull request?
<!--
Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->
This PR intends to add a new SQL config for controlling a plan explain mode in the events of (e.g., `SparkListenerSQLExecutionStart` and `SparkListenerSQLAdaptiveExecutionUpdate`) SQL listeners. In the current master, the output of `QueryExecution.toString` (this is equivalent to the "extended" explain mode) is stored in these events. I think it is useful to control the content via `SQLConf`. For example, the query "Details" content (TPCDS q66 query) of a SQL tab in a Spark web UI is changed as follows;
Before this PR:
After this PR:
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->
For better usability.
### Does this PR introduce any user-facing change?
<!--
If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
If no, write 'No'.
-->
Yes; since Spark 3.1, the `formatted` query explain mode is used in the events (e.g., `SparkListenerSQLExecutionStart` and `SparkListenerSQLAdaptiveExecutionUpdate`) of SQL listeners. To restore the behavior before Spark 3.0, you can set `spark.sql.queryDescriptionModeInListeners` to `extended`.
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why it was difficult to add.
-->
Added unit tests.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607776226
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607620214
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-608129642
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607776242
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120721/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28097: [SPARK-31325][SQL][Web
UI] Control a plan explain mode in the events of SQL listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607661651
**[Test build #120709 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120709/testReport)** for PR 28097 at commit [`0cad76a`](https://github.com/apache/spark/commit/0cad76abbd6c6bf214d020110daea4afb8a4e940).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607661712
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#discussion_r402470496
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala
##########
@@ -845,4 +845,41 @@ class AdaptiveQueryExecSuite
}
}
}
+
+ test("control a plan explain mode in listeners via SQLConf") {
+
+ def checkPlanDescription(mode: String, expected: Seq[String]): Unit = {
+ var checkDone = false
+ val listener = new SparkListener {
+ override def onOtherEvent(event: SparkListenerEvent): Unit = {
+ event match {
+ case SparkListenerSQLAdaptiveExecutionUpdate(_, planDescription, _) =>
+ assert(expected.forall(planDescription.contains))
+ checkDone = true
+ case _ => // ignore other events
+ }
+ }
+ }
+ spark.sparkContext.addSparkListener(listener)
+ withSQLConf(SQLConf.UI_EXPLAIN_MODE.key -> mode,
+ SQLConf.ADAPTIVE_EXECUTION_ENABLED.key -> "true",
+ SQLConf.AUTO_BROADCASTJOIN_THRESHOLD.key -> "80") {
+ val dfApdaptive = sql("SELECT * FROM testData join testData2 ON key = a where value = '1'")
Review comment:
nit. `WHERE`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28097: [SPARK-31325][SQL][Web
UI] Control a plan explain mode in the events of SQL listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607661070
**[Test build #120714 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120714/testReport)** for PR 28097 at commit [`3c684a5`](https://github.com/apache/spark/commit/3c684a5f58407ff42371d8a003254f46e0566fc2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on issue #28097: [SPARK-31325][SQL][Web
UI] Control a plan explain mode in the events of SQL listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
maropu commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-608214450
Thanks, @gengliangwang !
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on issue #28097: [SPARK-31325][SQL][Web
UI] Control a plan explain mode in the events of SQL listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
maropu commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607620515
@dongjoon-hyun @cloud-fan @gengliangwang How about this update?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28097: [SPARK-31325][SQL][Web
UI] Control a plan explain mode in the events of SQL listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-608129258
**[Test build #120740 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120740/testReport)** for PR 28097 at commit [`e950c41`](https://github.com/apache/spark/commit/e950c414a0119836ffffcdf4048b9ab3fc14435b).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607661892
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120709/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607661892
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120709/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607798860
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607800148
**[Test build #120722 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120722/testReport)** for PR 28097 at commit [`6ba5d18`](https://github.com/apache/spark/commit/6ba5d187436ea0d307ff8bfd5780539a57084548).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607787822
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120714/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607661637
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25412/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607800874
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25420/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#discussion_r402466406
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -2335,6 +2335,17 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
+ val UI_EXPLAIN_MODE = buildConf("spark.sql.ui.explainMode")
+ .doc("Configures the query explain mode used in the Spark SQL UI. The value can be 'simple', " +
+ "'extended', 'codegen', 'cost', or 'formatted'. The default log level is 'formatted'.")
+ .version("3.1.0")
+ .stringConf
+ .transform(_.toUpperCase(Locale.ROOT))
+ .checkValue(mode => Set("SIMPLE", "EXTENDED", "CODEGEN", "COST", "FORMATTED").contains(mode),
+ "Invalid value for 'spark.sql.queryDescriptionModeInListeners'. Valid values are " +
Review comment:
This seems to be a leftover after conf renaming. Maybe, `spark.sql.ui.explainMode`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#discussion_r402084110
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -2335,6 +2335,17 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
+ val QUERY_DESCRIPTION_MODE_IN_LISTENERS = buildConf("spark.sql.queryDescriptionModeInListeners")
Review comment:
Ur, yea. That looks nice. I'll update.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607652867
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#discussion_r402086157
##########
File path: docs/sql-migration-guide.md
##########
@@ -23,8 +23,11 @@ license: |
{:toc}
## Upgrading from Spark SQL 3.0 to 3.1
+
Review comment:
It seems the other parts (e.g., `## Upgrading from Spark SQL 2.4.4 to 2.4.5`) have a single blank line in the head. So, I added a blank here, too.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607658706
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#discussion_r402081054
##########
File path: docs/sql-migration-guide.md
##########
@@ -23,8 +23,11 @@ license: |
{:toc}
## Upgrading from Spark SQL 3.0 to 3.1
+
- Since Spark 3.1, grouping_id() returns long values. In Spark version 3.0 and earlier, this function returns int values. To restore the behavior before Spark 3.0, you can set `spark.sql.legacy.integerGroupingId` to `true`.
+ - Since Spark 3.1, the `formatted` query explain mode is used in the events (e.g., `SparkListenerSQLExecutionStart` and `SparkListenerSQLAdaptiveExecutionUpdate`) of SQL listeners. To restore the behavior before Spark 3.0, you can set `spark.sql.queryDescriptionModeInListeners` to `extended`.
Review comment:
Ur, yea. I added this because I thought user applications could listen to `SparkListenerSQLExecutionStart` and this PR changed the content in the event. But, I don't have a strong opinion on this, so I'll remove it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#discussion_r402174140
##########
File path: docs/sql-migration-guide.md
##########
@@ -23,8 +23,11 @@ license: |
{:toc}
## Upgrading from Spark SQL 3.0 to 3.1
+
Review comment:
A space between `##` and `-` is actually a required change for the correct MD syntax IIRC. Morden tools support no spaces between them, but technically it's required for strict MD syntax.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607619951
**[Test build #120709 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120709/testReport)** for PR 28097 at commit [`0cad76a`](https://github.com/apache/spark/commit/0cad76abbd6c6bf214d020110daea4afb8a4e940).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607620219
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25407/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #28097: [SPARK-31325][SQL][Web
UI] Control a plan explain mode in the events of SQL listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607648720
LGTM! A very useful feature!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28097: [SPARK-31325][SQL][Web
UI] Control a plan explain mode in the events of SQL listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607786968
**[Test build #120714 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120714/testReport)** for PR 28097 at commit [`3c684a5`](https://github.com/apache/spark/commit/3c684a5f58407ff42371d8a003254f46e0566fc2).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#discussion_r402086061
##########
File path: docs/sql-migration-guide.md
##########
@@ -23,8 +23,11 @@ license: |
{:toc}
## Upgrading from Spark SQL 3.0 to 3.1
+
- Since Spark 3.1, grouping_id() returns long values. In Spark version 3.0 and earlier, this function returns int values. To restore the behavior before Spark 3.0, you can set `spark.sql.legacy.integerGroupingId` to `true`.
+ - Since Spark 3.1, the `formatted` query explain mode is used in the events (e.g., `SparkListenerSQLExecutionStart` and `SparkListenerSQLAdaptiveExecutionUpdate`) of SQL listeners. To restore the behavior before Spark 3.0, you can set `spark.sql.queryDescriptionModeInListeners` to `extended`.
Review comment:
We only add a migration guide if we change behavior BY DEFAULT.
We can mention it in release notes though, to promote this feature.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607620214
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607668210
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25414/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607798860
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28097: [SPARK-31325][SQL][Web
UI] Control a plan explain mode in the events of SQL listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607619951
**[Test build #120709 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120709/testReport)** for PR 28097 at commit [`0cad76a`](https://github.com/apache/spark/commit/0cad76abbd6c6bf214d020110daea4afb8a4e940).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#discussion_r402194965
##########
File path: docs/sql-migration-guide.md
##########
@@ -23,8 +23,11 @@ license: |
{:toc}
## Upgrading from Spark SQL 3.0 to 3.1
+
Review comment:
Ah, I see.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607652875
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25410/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#discussion_r402081659
##########
File path: docs/sql-migration-guide.md
##########
@@ -23,8 +23,11 @@ license: |
{:toc}
## Upgrading from Spark SQL 3.0 to 3.1
+
- Since Spark 3.1, grouping_id() returns long values. In Spark version 3.0 and earlier, this function returns int values. To restore the behavior before Spark 3.0, you can set `spark.sql.legacy.integerGroupingId` to `true`.
+ - Since Spark 3.1, the `formatted` query explain mode is used in the events (e.g., `SparkListenerSQLExecutionStart` and `SparkListenerSQLAdaptiveExecutionUpdate`) of SQL listeners. To restore the behavior before Spark 3.0, you can set `spark.sql.queryDescriptionModeInListeners` to `extended`.
Review comment:
It's sort of behavior change and it worth mentioning the new configuration. I think it is fine to have the migration guide.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607787807
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#discussion_r402475301
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -2335,6 +2335,17 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
+ val UI_EXPLAIN_MODE = buildConf("spark.sql.ui.explainMode")
+ .doc("Configures the query explain mode used in the Spark SQL UI. The value can be 'simple', " +
+ "'extended', 'codegen', 'cost', or 'formatted'. The default log level is 'formatted'.")
+ .version("3.1.0")
+ .stringConf
+ .transform(_.toUpperCase(Locale.ROOT))
+ .checkValue(mode => Set("SIMPLE", "EXTENDED", "CODEGEN", "COST", "FORMATTED").contains(mode),
+ "Invalid value for 'spark.sql.queryDescriptionModeInListeners'. Valid values are " +
+ "'simple', 'extended', 'codegen', 'cost' and 'formatted'.")
Review comment:
indentation?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28097: [SPARK-31325][SQL][Web
UI] Control a plan explain mode in the events of SQL listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-608201285
**[Test build #120740 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120740/testReport)** for PR 28097 at commit [`e950c41`](https://github.com/apache/spark/commit/e950c414a0119836ffffcdf4048b9ab3fc14435b).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-608129647
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25438/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28097: [SPARK-31325][SQL][Web
UI] Control a plan explain mode in the events of SQL listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607658303
**[Test build #120713 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120713/testReport)** for PR 28097 at commit [`750ad42`](https://github.com/apache/spark/commit/750ad42a2d223c48c90790cd046d58cf340ea94e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607661883
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607949582
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607655263
**[Test build #120712 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120712/testReport)** for PR 28097 at commit [`35c8e5d`](https://github.com/apache/spark/commit/35c8e5d1a2da3b1b0ee6897932ab54ebef4f1f60).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on issue #28097: [SPARK-31325][SQL][Web
UI] Control a plan explain mode in the events of SQL listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
maropu commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607650458
Thanks, all! I'll address the @cloud-fan comment now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28097: [SPARK-31325][SQL][Web
UI] Control a plan explain mode in the events of SQL listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607776152
**[Test build #120721 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120721/testReport)** for PR 28097 at commit [`6ba5d18`](https://github.com/apache/spark/commit/6ba5d187436ea0d307ff8bfd5780539a57084548).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607798873
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120716/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607664882
retest this please.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28097: [SPARK-31325][SQL][Web
UI] Control a plan explain mode in the events of SQL listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607661652
**[Test build #120713 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120713/testReport)** for PR 28097 at commit [`750ad42`](https://github.com/apache/spark/commit/750ad42a2d223c48c90790cd046d58cf340ea94e).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607620219
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25407/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-608129258
**[Test build #120740 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120740/testReport)** for PR 28097 at commit [`e950c41`](https://github.com/apache/spark/commit/e950c414a0119836ffffcdf4048b9ab3fc14435b).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #28097: [SPARK-31325][SQL][Web
UI] Control a plan explain mode in the events of SQL listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607799550
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang closed pull request #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
gengliangwang closed pull request #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607800857
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607787822
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120714/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on issue #28097: [SPARK-31325][SQL][Web
UI] Control a plan explain mode in the events of SQL listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
maropu commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-608128951
Thanks for the reviews, @dongjoon-hyun . Plz check again?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607751239
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607661637
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25412/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28097: [SPARK-31325][SQL][Web
UI] Control a plan explain mode in the events of SQL listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607797891
**[Test build #120716 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120716/testReport)** for PR 28097 at commit [`3c684a5`](https://github.com/apache/spark/commit/3c684a5f58407ff42371d8a003254f46e0566fc2).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-608129642
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607652875
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25410/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607661692
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607776226
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607661712
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607751239
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#discussion_r402076384
##########
File path: docs/sql-migration-guide.md
##########
@@ -23,8 +23,11 @@ license: |
{:toc}
## Upgrading from Spark SQL 3.0 to 3.1
+
Review comment:
nit: unneeded change?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607661070
**[Test build #120714 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120714/testReport)** for PR 28097 at commit [`3c684a5`](https://github.com/apache/spark/commit/3c684a5f58407ff42371d8a003254f46e0566fc2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-608201758
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120740/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#discussion_r402477030
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -2335,6 +2335,17 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
+ val UI_EXPLAIN_MODE = buildConf("spark.sql.ui.explainMode")
+ .doc("Configures the query explain mode used in the Spark SQL UI. The value can be 'simple', " +
+ "'extended', 'codegen', 'cost', or 'formatted'. The default log level is 'formatted'.")
Review comment:
Shall we remove the word, `log`? `The default level` might be enough.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607949595
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120722/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607800857
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607661715
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120713/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#discussion_r402078044
##########
File path: docs/sql-migration-guide.md
##########
@@ -23,8 +23,11 @@ license: |
{:toc}
## Upgrading from Spark SQL 3.0 to 3.1
+
- Since Spark 3.1, grouping_id() returns long values. In Spark version 3.0 and earlier, this function returns int values. To restore the behavior before Spark 3.0, you can set `spark.sql.legacy.integerGroupingId` to `true`.
+ - Since Spark 3.1, the `formatted` query explain mode is used in the events (e.g., `SparkListenerSQLExecutionStart` and `SparkListenerSQLAdaptiveExecutionUpdate`) of SQL listeners. To restore the behavior before Spark 3.0, you can set `spark.sql.queryDescriptionModeInListeners` to `extended`.
Review comment:
this doesn't break anything but just adds a new feature, do we really need a migration guide?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607658706
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-608201758
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120740/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607751250
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25419/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607776242
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120721/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu edited a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
maropu edited a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607746903
Thanks for the reviews, all! pending, Jenkins.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607658713
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25411/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang edited a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
gengliangwang edited a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-608210759
Thanks, I will merge into master after the github PR builder pass.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607661692
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607949595
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120722/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#discussion_r402078311
##########
File path: docs/sql-migration-guide.md
##########
@@ -23,8 +23,11 @@ license: |
{:toc}
## Upgrading from Spark SQL 3.0 to 3.1
+
- Since Spark 3.1, grouping_id() returns long values. In Spark version 3.0 and earlier, this function returns int values. To restore the behavior before Spark 3.0, you can set `spark.sql.legacy.integerGroupingId` to `true`.
+ - Since Spark 3.1, the `formatted` query explain mode is used in the events (e.g., `SparkListenerSQLExecutionStart` and `SparkListenerSQLAdaptiveExecutionUpdate`) of SQL listeners. To restore the behavior before Spark 3.0, you can set `spark.sql.queryDescriptionModeInListeners` to `extended`.
Review comment:
I think it is too internal to talk about the SQL listener and events like `SparkListenerSQLExecutionStart` here.
How about
```
SQL UI data adopts the `formatted` mode for the query plan explain results.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28097: [SPARK-31325][SQL][Web
UI] Control a plan explain mode in the events of SQL listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607655263
**[Test build #120712 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120712/testReport)** for PR 28097 at commit [`35c8e5d`](https://github.com/apache/spark/commit/35c8e5d1a2da3b1b0ee6897932ab54ebef4f1f60).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607652867
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607668194
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28097: [SPARK-31325][SQL][Web
UI] Control a plan explain mode in the events of SQL listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607661648
**[Test build #120712 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120712/testReport)** for PR 28097 at commit [`35c8e5d`](https://github.com/apache/spark/commit/35c8e5d1a2da3b1b0ee6897932ab54ebef4f1f60).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607751250
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25419/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#discussion_r402079715
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -2335,6 +2335,17 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
+ val QUERY_DESCRIPTION_MODE_IN_LISTENERS = buildConf("spark.sql.queryDescriptionModeInListeners")
Review comment:
How about
```
spark.sql.ui.explainMode
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28097: [SPARK-31325][SQL][Web
UI] Control a plan explain mode in the events of SQL listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607948487
**[Test build #120722 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120722/testReport)** for PR 28097 at commit [`6ba5d18`](https://github.com/apache/spark/commit/6ba5d187436ea0d307ff8bfd5780539a57084548).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-608201752
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-608129647
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25438/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607649204
@maropu +1 for the proposal!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu edited a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
maropu edited a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607746903
Thanks for the reviews, @HyukjinKwon @cloud-fan @gengliangwang ! pending, Jenkins.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607661704
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120712/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607661704
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120712/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607750749
**[Test build #120721 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120721/testReport)** for PR 28097 at commit [`6ba5d18`](https://github.com/apache/spark/commit/6ba5d187436ea0d307ff8bfd5780539a57084548).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607661633
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#discussion_r402474058
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListenerSuite.scala
##########
@@ -317,6 +317,40 @@ class SQLAppStatusListenerSuite extends SharedSparkSession with JsonTestUtils
checkAnswer(statusStore.executionMetrics(executionId), accumulatorUpdates.mapValues(_ * 11))
}
+ test("control a plan explain mode in listeners via SQLConf") {
+
+ def checkPlanDescription(mode: String, expected: Seq[String]): Unit = {
+ var checkDone = false
+ val listener = new SparkListener {
+ override def onOtherEvent(event: SparkListenerEvent): Unit = {
+ event match {
+ case SparkListenerSQLExecutionStart(_, _, _, planDescription, _, _) =>
+ assert(expected.forall(planDescription.contains))
+ checkDone = true
+ case _ => // ignore other events
+ }
+ }
+ }
+ spark.sparkContext.addSparkListener(listener)
+ withSQLConf(SQLConf.UI_EXPLAIN_MODE.key -> mode) {
+ createTestDataFrame.collect()
+ spark.sparkContext.listenerBus.waitUntilEmpty()
+ assert(checkDone)
+ spark.sparkContext.removeSparkListener(listener)
Review comment:
Do we need to use `finally` to isolate the failure at line 338 from the other UTs?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607979200
Thank you for pinging me, @maropu . This looks super-useful. I added a few minor comments.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607658713
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25411/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607667622
**[Test build #120716 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120716/testReport)** for PR 28097 at commit [`3c684a5`](https://github.com/apache/spark/commit/3c684a5f58407ff42371d8a003254f46e0566fc2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on issue #28097: [SPARK-31325][SQL][Web
UI] Control a plan explain mode in the events of SQL listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
maropu commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607746903
Thanks for the reviews, all! Pending, Jenkins.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28097: [SPARK-31325][SQL][Web
UI] Control a plan explain mode in the events of SQL listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607800148
**[Test build #120722 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120722/testReport)** for PR 28097 at commit [`6ba5d18`](https://github.com/apache/spark/commit/6ba5d187436ea0d307ff8bfd5780539a57084548).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607661633
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607668210
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25414/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-608201752
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607949582
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#discussion_r402465978
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -2335,6 +2335,17 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
+ val UI_EXPLAIN_MODE = buildConf("spark.sql.ui.explainMode")
+ .doc("Configures the query explain mode used in the Spark SQL UI. The value can be 'simple', " +
+ "'extended', 'codegen', 'cost', or 'formatted'. The default log level is 'formatted'.")
+ .version("3.1.0")
+ .stringConf
+ .transform(_.toUpperCase(Locale.ROOT))
+ .checkValue(mode => Set("SIMPLE", "EXTENDED", "CODEGEN", "COST", "FORMATTED").contains(mode),
+ "Invalid value for 'spark.sql.queryDescriptionModeInListeners'. Valid values are " +
Review comment:
Please rename this together, @maropu .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607658303
**[Test build #120713 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120713/testReport)** for PR 28097 at commit [`750ad42`](https://github.com/apache/spark/commit/750ad42a2d223c48c90790cd046d58cf340ea94e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607787807
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#discussion_r402634528
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala
##########
@@ -845,4 +845,41 @@ class AdaptiveQueryExecSuite
}
}
}
+
+ test("control a plan explain mode in listeners via SQLConf") {
+
+ def checkPlanDescription(mode: String, expected: Seq[String]): Unit = {
+ var checkDone = false
+ val listener = new SparkListener {
+ override def onOtherEvent(event: SparkListenerEvent): Unit = {
+ event match {
+ case SparkListenerSQLAdaptiveExecutionUpdate(_, planDescription, _) =>
+ assert(expected.forall(planDescription.contains))
+ checkDone = true
+ case _ => // ignore other events
+ }
+ }
+ }
+ spark.sparkContext.addSparkListener(listener)
+ withSQLConf(SQLConf.UI_EXPLAIN_MODE.key -> mode,
+ SQLConf.ADAPTIVE_EXECUTION_ENABLED.key -> "true",
+ SQLConf.AUTO_BROADCASTJOIN_THRESHOLD.key -> "80") {
+ val dfApdaptive = sql("SELECT * FROM testData join testData2 ON key = a where value = '1'")
Review comment:
Ah, yea. Also, capitalized `join`, too.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607668194
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28097: [SPARK-31325][SQL][Web
UI] Control a plan explain mode in the events of SQL listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607667622
**[Test build #120716 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120716/testReport)** for PR 28097 at commit [`3c684a5`](https://github.com/apache/spark/commit/3c684a5f58407ff42371d8a003254f46e0566fc2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607661883
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-608210759
Thanks, merging to master
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607800874
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25420/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28097: [SPARK-31325][SQL][Web
UI] Control a plan explain mode in the events of SQL listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607750749
**[Test build #120721 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120721/testReport)** for PR 28097 at commit [`6ba5d18`](https://github.com/apache/spark/commit/6ba5d187436ea0d307ff8bfd5780539a57084548).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607798873
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120716/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28097:
[SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL
listeners via SQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28097: [SPARK-31325][SQL][Web UI] Control a plan explain mode in the events of SQL listeners via SQLConf
URL: https://github.com/apache/spark/pull/28097#issuecomment-607661715
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120713/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org