You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/02/01 02:29:37 UTC
[GitHub] [spark] xuanyuanking opened a new pull request #27425:
[SPARK-29543][SS][Follow Up] The configs in Structured Streaming UI should
be put into StaticSQLConf
xuanyuanking opened a new pull request #27425: [SPARK-29543][SS][Follow Up] The configs in Structured Streaming UI should be put into StaticSQLConf
URL: https://github.com/apache/spark/pull/27425
### What changes were proposed in this pull request?
Put the configs below needed by Structured Streaming UI into StaticSQLConf:
- spark.sql.streaming.ui.enabled
- spark.sql.streaming.ui.retainedProgressUpdates
- spark.sql.streaming.ui.retainedQueries
### Why are the changes needed?
Make all SS UI configs consistent with other similar configs in usage and naming.
### Does this PR introduce any user-facing change?
Yes, add new static config `spark.sql.streaming.ui.retainedProgressUpdates`.
### How was this patch tested?
Existing UT.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-581002533
**[Test build #117709 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117709/testReport)** for PR 27425 at commit [`e4c3b38`](https://github.com/apache/spark/commit/e4c3b388f8be051f3ef619f08a636975d1156d0f).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-581123512
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27425: [SPARK-29543][SS][Follow
Up] The configs in Structured Streaming UI should be put into StaticSQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27425: [SPARK-29543][SS][Follow Up] The configs in Structured Streaming UI should be put into StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-580981804
**[Test build #117709 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117709/testReport)** for PR 27425 at commit [`e4c3b38`](https://github.com/apache/spark/commit/e4c3b388f8be051f3ef619f08a636975d1156d0f).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-581002667
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on a change in pull request #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#discussion_r373836476
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala
##########
@@ -146,12 +146,14 @@ private[sql] class SharedState(
*/
lazy val streamingQueryStatusListener: Option[StreamingQueryStatusListener] = {
val sqlConf = SQLConf.get
Review comment:
Thanks, then StreamingQueryStatusListener need a little change, done in d763abb.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27425:
[SPARK-29543][SS][Follow Up] The configs in Structured Streaming UI should
be put into StaticSQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27425: [SPARK-29543][SS][Follow Up] The configs in Structured Streaming UI should be put into StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-580981926
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22469/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-581123512
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zsxwing commented on a change in pull request #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
zsxwing commented on a change in pull request #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#discussion_r373824863
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/StaticSQLConf.scala
##########
@@ -176,4 +176,23 @@ object StaticSQLConf {
.internal()
.booleanConf
.createWithDefault(true)
+
+ val STREAMING_UI_ENABLED =
+ buildStaticConf("spark.sql.streaming.ui.enabled")
+ .doc("Whether to run the structured streaming UI for the Spark application.")
+ .booleanConf
+ .createWithDefault(true)
+
+ val STREAMING_UI_RETAINED_PROGRESS_UPDATES =
+ buildStaticConf("spark.sql.streaming.ui.retainedProgressUpdates")
+ .doc("The number of progress updates to retain for a streaming query for structured " +
Review comment:
nit: `structured streaming ui` -> `Structured Streaming UI`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-581123961
**[Test build #117740 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117740/testReport)** for PR 27425 at commit [`d763abb`](https://github.com/apache/spark/commit/d763abb6368c25b2047a30ec00295752283f71d2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on issue #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on issue #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-581280423
Thanks for the review!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-581002668
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117709/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-581123516
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22501/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zsxwing commented on a change in pull request #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
zsxwing commented on a change in pull request #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#discussion_r373824921
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala
##########
@@ -146,12 +146,14 @@ private[sql] class SharedState(
*/
lazy val streamingQueryStatusListener: Option[StreamingQueryStatusListener] = {
val sqlConf = SQLConf.get
Review comment:
nit: we can remove this and use `conf` in this class directly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27425:
[SPARK-29543][SS][Follow Up] The configs in Structured Streaming UI should
be put into StaticSQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27425: [SPARK-29543][SS][Follow Up] The configs in Structured Streaming UI should be put into StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-580981926
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22469/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-580981804
**[Test build #117709 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117709/testReport)** for PR 27425 at commit [`e4c3b38`](https://github.com/apache/spark/commit/e4c3b388f8be051f3ef619f08a636975d1156d0f).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zsxwing commented on a change in pull request #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
zsxwing commented on a change in pull request #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#discussion_r373824802
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/StaticSQLConf.scala
##########
@@ -176,4 +176,23 @@ object StaticSQLConf {
.internal()
.booleanConf
.createWithDefault(true)
+
+ val STREAMING_UI_ENABLED =
+ buildStaticConf("spark.sql.streaming.ui.enabled")
+ .doc("Whether to run the structured streaming UI for the Spark application.")
Review comment:
nit: `Whether to run the Structured Streaming Web UI for the Spark application when the Spark Web UI is enabled.`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] uncleGen commented on issue #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
uncleGen commented on issue #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-581200735
Thanks for this work, LGTM.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zsxwing commented on issue #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
zsxwing commented on issue #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-581276861
Thanks! Merging to master and branch-3.0.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27425:
[SPARK-29543][SS][Follow Up] The configs in Structured Streaming UI should
be put into StaticSQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27425: [SPARK-29543][SS][Follow Up] The configs in Structured Streaming UI should be put into StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-580981924
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-581144202
**[Test build #117740 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117740/testReport)** for PR 27425 at commit [`d763abb`](https://github.com/apache/spark/commit/d763abb6368c25b2047a30ec00295752283f71d2).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-581144371
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117740/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on issue #27425:
[SPARK-29543][SS][Follow Up] The configs in Structured Streaming UI should
be put into StaticSQLConf
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on issue #27425: [SPARK-29543][SS][Follow Up] The configs in Structured Streaming UI should be put into StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-580981748
cc @zsxwing @uncleGen
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zsxwing commented on a change in pull request #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
zsxwing commented on a change in pull request #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#discussion_r373824898
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/StaticSQLConf.scala
##########
@@ -176,4 +176,23 @@ object StaticSQLConf {
.internal()
.booleanConf
.createWithDefault(true)
+
+ val STREAMING_UI_ENABLED =
+ buildStaticConf("spark.sql.streaming.ui.enabled")
+ .doc("Whether to run the structured streaming UI for the Spark application.")
+ .booleanConf
+ .createWithDefault(true)
+
+ val STREAMING_UI_RETAINED_PROGRESS_UPDATES =
+ buildStaticConf("spark.sql.streaming.ui.retainedProgressUpdates")
+ .doc("The number of progress updates to retain for a streaming query for structured " +
+ "streaming ui.")
+ .intConf
+ .createWithDefault(100)
+
+ val STREAMING_UI_RETAINED_QUERIES =
+ buildStaticConf("spark.sql.streaming.ui.retainedQueries")
+ .doc("The number of inactive queries to retain for structured streaming ui.")
Review comment:
ditto
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-581144371
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117740/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-581123516
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22501/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27425:
[SPARK-29543][SS][Follow Up] The configs in Structured Streaming UI should
be put into StaticSQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27425: [SPARK-29543][SS][Follow Up] The configs in Structured Streaming UI should be put into StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-580981924
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on a change in pull request #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#discussion_r373836452
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/StaticSQLConf.scala
##########
@@ -176,4 +176,23 @@ object StaticSQLConf {
.internal()
.booleanConf
.createWithDefault(true)
+
+ val STREAMING_UI_ENABLED =
+ buildStaticConf("spark.sql.streaming.ui.enabled")
+ .doc("Whether to run the structured streaming UI for the Spark application.")
+ .booleanConf
+ .createWithDefault(true)
+
+ val STREAMING_UI_RETAINED_PROGRESS_UPDATES =
+ buildStaticConf("spark.sql.streaming.ui.retainedProgressUpdates")
+ .doc("The number of progress updates to retain for a streaming query for structured " +
+ "streaming ui.")
+ .intConf
+ .createWithDefault(100)
+
+ val STREAMING_UI_RETAINED_QUERIES =
+ buildStaticConf("spark.sql.streaming.ui.retainedQueries")
+ .doc("The number of inactive queries to retain for structured streaming ui.")
Review comment:
Thanks, doc changes done in d763abb.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-581144368
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-581002667
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] asfgit closed pull request #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-581123961
**[Test build #117740 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117740/testReport)** for PR 27425 at commit [`d763abb`](https://github.com/apache/spark/commit/d763abb6368c25b2047a30ec00295752283f71d2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-581144368
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27425:
[SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to
StaticSQLConf
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27425: [SPARK-29543][SS][FOLLOWUP] Move `spark.sql.streaming.ui.*` configs to StaticSQLConf
URL: https://github.com/apache/spark/pull/27425#issuecomment-581002668
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117709/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org