You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by nblintao <gi...@git.apache.org> on 2016/07/22 15:25:23 UTC
[GitHub] spark pull request #14319: [SPARK-16635][WEBUI][SQL] Provide Session support...
GitHub user nblintao opened a pull request:
https://github.com/apache/spark/pull/14319
[SPARK-16635][WEBUI][SQL] Provide Session support in the Spark UI
## What changes were proposed in this pull request?
Add a Session Tab and the related Session Page in the Spark UI.
The page could show all properties configured for the session. It could be updated to the latest configuration after refreshing, compared to the SparkContext configurations showed in the Environments Page which cannot be updated during runtime.
## How was this patch tested?
Manually tested and viewed, and unit tests.
When a `spark-shell` is opened, the Session page could be found at the related web UI (e.g. http://192.168.1.104:4040/session/), showing all the properties of the session.
First input lines below into the shell, a new row about "Property A" could be found at the Session page after refreshing.
```scala
import org.apache.spark.sql.SparkSession
SparkSession.builder.config("Property A", "the value of Property A").getOrCreate()
```
Then input the line below, another row is shown.
```scala
spark.conf.set("Property B", "the value of Property B")
```
Screenshot:
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nblintao/spark sessionSupport
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14319.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14319
----
commit f7741a0d343ec633a38316e489b709e84d1ccf96
Author: Tao Lin <nb...@gmail.com>
Date: 2016-07-22T15:04:03Z
add session support in the web UI
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14319: [SPARK-16635] [WEBUI] [SQL] [WIP] Provide Session...
Posted by nblintao <gi...@git.apache.org>.
Github user nblintao closed the pull request at:
https://github.com/apache/spark/pull/14319
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14319: [SPARK-16635][WEBUI][SQL] Provide Session support in the...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14319
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14319: [SPARK-16635][WEBUI][SQL] Provide Session support in the...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14319
**[Test build #62731 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62731/consoleFull)** for PR 14319 at commit [`f7741a0`](https://github.com/apache/spark/commit/f7741a0d343ec633a38316e489b709e84d1ccf96).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14319: [SPARK-16635] [WEBUI] [SQL] [WIP] Provide Session suppor...
Posted by nblintao <gi...@git.apache.org>.
Github user nblintao commented on the issue:
https://github.com/apache/spark/pull/14319
Doing more study to figure out the requirements on providing per-session information to users. So, I decided to close the PR for now. The work will be continued after figuring out how this page should be designed.
Thank @yhuai, @ajbozarth, and others involved.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14319: [SPARK-16635][WEBUI][SQL] Provide Session support in the...
Posted by ajbozarth <gi...@git.apache.org>.
Github user ajbozarth commented on the issue:
https://github.com/apache/spark/pull/14319
I've read through your code and didn't catch any issues, I also checked it out and it looks good. I think this is a nice feature to add, my only qualm is it add yet another tab to the Web UI. If everyone is ok adding another tab I'm ok with it, but could it make sense to just put the table in the env tab? Opinions? @srowen @tgravescs
Also I think you missed something in your updates to the tests but I couldn't see it on first look
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14319: [SPARK-16635][WEBUI][SQL] Provide Session support in the...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14319
**[Test build #62731 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62731/consoleFull)** for PR 14319 at commit [`f7741a0`](https://github.com/apache/spark/commit/f7741a0d343ec633a38316e489b709e84d1ccf96).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `case class SparkListenerSessionUpdate(sessionDetails: Map[String, String])`
* `class SessionListener extends SparkListener `
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14319: [SPARK-16635] [WEBUI] [SQL] [WIP] Provide Session suppor...
Posted by nblintao <gi...@git.apache.org>.
Github user nblintao commented on the issue:
https://github.com/apache/spark/pull/14319
Thanks, @ajbozarth.
Yes, I think the configuration alone is not enough for a new tab. @yhuai and I actually plan to do more on this tab. As mentioned in [JIRA](https://issues.apache.org/jira/browse/SPARK-16635):
> In Spark 2.0, SparkSession will be the entry point of spark. We can think about how to display per-session info (like configurations) and jobs in a good way. We can start to experiment a Session tab and see if we can come up with a way to just show a single session's info in a nice way. Then, we can see if this new tab can replace any existing tab.
So this is not final and please make more suggestions. I've just put WIP in the title to avoid misleading.
Also, I will try to fix the error recently.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14319: [SPARK-16635][WEBUI][SQL] Provide Session support in the...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14319
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62731/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org