You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by suyanNone <gi...@git.apache.org> on 2016/04/21 11:04:20 UTC
[GitHub] spark pull request: [SPARK-14750][Spark][UI] Support spark on yarn...
GitHub user suyanNone opened a pull request:
https://github.com/apache/spark/pull/12570
[SPARK-14750][Spark][UI] Support spark on yarn user refer finished application log in historyServer
## What changes were proposed in this pull request?
For spark on yarn, make user can refer application logs while application finished.
## How was this patch tested?
manual tests
before：
![Uploading Screenshot from 2016-04-21 16:59:45.png…]()
after
![Uploading Screenshot from 2016-04-21 17:00:44.png…]()
![Uploading Screenshot from 2016-04-21 17:01:15.png…]()
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/suyanNone/spark refer-hdfs
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12570.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12570
----
commit f37d5c0ad8a0fa5eaa7c305c6ddad9d6702e462c
Author: hushan <hu...@xiaomi.com>
Date: 2016-04-20T08:28:38Z
[SPARK][UI] Make historyserver refer application log
Summary: Ref T5959
Test Plan: Test local
Reviewers: liushaohui, zouchenjun, wangjiasheng, peng.zhang
Reviewed By: peng.zhang
Maniphest Tasks: T5959
Differential Revision: https://phabricator.d.xiaomi.net/D31886
Conflicts:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala
commit 80262effb0918616a7d33e81c8e97db5d56d74ab
Author: hushan <hu...@xiaomi.com>
Date: 2016-04-21T03:26:15Z
Refine
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14750][Spark][UI] Support spark on yarn...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12570#issuecomment-212921417
**[Test build #56539 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56539/consoleFull)** for PR 12570 at commit [`6c84b44`](https://github.com/apache/spark/commit/6c84b44d77166ae0a15b4d66dc88635fde398248).
* This patch **fails Scala style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14750][Spark][UI] Support spark on yarn...
Posted by suyanNone <gi...@git.apache.org>.
Github user suyanNone closed the pull request at:
https://github.com/apache/spark/pull/12570
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14750][Spark][UI] Support spark on yarn...
Posted by suyanNone <gi...@git.apache.org>.
Github user suyanNone commented on the pull request:
https://github.com/apache/spark/pull/12570#issuecomment-218412962
@ajbozarth oh...I not familiar with standalone... I have no idea about there had a class named `LogPage`, nice for you to tell me about that...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14750][Spark][UI] Support spark on yarn...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12570#issuecomment-212822666
**[Test build #56520 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56520/consoleFull)** for PR 12570 at commit [`80262ef`](https://github.com/apache/spark/commit/80262effb0918616a7d33e81c8e97db5d56d74ab).
* This patch **fails Scala style tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `class YarnExecutorLogsPage(parent: ExecutorsTab) extends WebUIPage(\"executorLogs\") with Logging `
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14750][Spark][UI] Support spark on yarn...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/12570#issuecomment-215948612
Yes, I think this PR should be closed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14750][Spark][UI] Support spark on yarn...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12570#issuecomment-212822669
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14750][Spark][UI] Support spark on yarn...
Posted by ajbozarth <gi...@git.apache.org>.
Github user ajbozarth commented on the pull request:
https://github.com/apache/spark/pull/12570#issuecomment-213140513
From what I can tell you reimplemented `LogPage` for yarn specifically. Wouldn't it make more sense to extend and surface `LogPage` for your use case? I've read through your code a bit and it looks like you have some good improvements over certain parts of `LogPage` but I think creating a completely separate log viewer for yarn is unnecessary and redundant. (Also note my other observations in the Jira.)
@srowen: you were commenting on the Jira, what do you think?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14750][Spark][UI] Support spark on yarn...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12570#issuecomment-212920876
**[Test build #56539 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56539/consoleFull)** for PR 12570 at commit [`6c84b44`](https://github.com/apache/spark/commit/6c84b44d77166ae0a15b4d66dc88635fde398248).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14750][Spark][UI] Support spark on yarn...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12570#issuecomment-212921424
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14750][Spark][UI] Support spark on yarn...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12570#issuecomment-212822671
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56520/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14750][Spark][UI] Support spark on yarn...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12570#issuecomment-212822370
**[Test build #56520 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56520/consoleFull)** for PR 12570 at commit [`80262ef`](https://github.com/apache/spark/commit/80262effb0918616a7d33e81c8e97db5d56d74ab).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-14750][Spark][UI] Support spark on yarn...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12570#issuecomment-212921427
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56539/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org