You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by tg...@apache.org on 2016/03/08 16:09:56 UTC
spark git commit: [SPARK-13675][UI] Fix wrong historyserver url link
for application running in yarn cluster mode
Repository: spark
Updated Branches:
refs/heads/master 9bf76ddde -> 9e86e6efd
[SPARK-13675][UI] Fix wrong historyserver url link for application running in yarn cluster mode
## What changes were proposed in this pull request?
Current URL for each application to access history UI is like:
http://localhost:18080/history/application_1457058760338_0016/1/jobs/ or http://localhost:18080/history/application_1457058760338_0016/2/jobs/
Here **1** or **2** represents the number of attempts in `historypage.js`, but it will parse to attempt id in `HistoryServer`, while the correct attempt id should be like "appattempt_1457058760338_0016_000002", so it will fail to parse to a correct attempt id in HistoryServer.
This is OK in yarn client mode, since we don't need this attempt id to fetch out the app cache, but it is failed in yarn cluster mode, where attempt id "1" or "2" is actually wrong.
So here we should fix this url to parse the correct application id and attempt id. Also the suffix "jobs/" is not needed.
Here is the screenshot:
## How was this patch tested?
This patch is tested manually, with different master and deploy mode.
Author: jerryshao <ss...@hortonworks.com>
Closes #11518 from jerryshao/SPARK-13675.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/9e86e6ef
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/9e86e6ef
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/9e86e6ef
Branch: refs/heads/master
Commit: 9e86e6efd136182bb00fa925c3818c9baccbd1fc
Parents: 9bf76dd
Author: jerryshao <ss...@hortonworks.com>
Authored: Tue Mar 8 09:09:42 2016 -0600
Committer: Tom Graves <tg...@yahoo-inc.com>
Committed: Tue Mar 8 09:09:42 2016 -0600
----------------------------------------------------------------------
.../spark/ui/static/historypage-template.html | 4 ++--
.../org/apache/spark/ui/static/historypage.js | 19 +++++++++++++++++--
2 files changed, 19 insertions(+), 4 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/9e86e6ef/core/src/main/resources/org/apache/spark/ui/static/historypage-template.html
----------------------------------------------------------------------
diff --git a/core/src/main/resources/org/apache/spark/ui/static/historypage-template.html b/core/src/main/resources/org/apache/spark/ui/static/historypage-template.html
index a2b3826..e5ed5b3 100644
--- a/core/src/main/resources/org/apache/spark/ui/static/historypage-template.html
+++ b/core/src/main/resources/org/apache/spark/ui/static/historypage-template.html
@@ -64,10 +64,10 @@
<tbody>
{{#applications}}
<tr>
- <td class="rowGroupColumn"><span title="{{id}}"><a href="/history/{{id}}/{{num}}/jobs/">{{id}}</a></span></td>
+ <td class="rowGroupColumn"><span title="{{id}}"><a href="{{url}}">{{id}}</a></span></td>
<td class="rowGroupColumn">{{name}}</td>
{{#attempts}}
- <td class="attemptIDSpan"><a href="/history/{{id}}/{{attemptId}}/jobs/">{{attemptId}}</a></td>
+ <td class="attemptIDSpan"><a href="/history/{{id}}/{{attemptId}}/">{{attemptId}}</a></td>
<td>{{startTime}}</td>
<td>{{endTime}}</td>
<td><span title="{{duration}}" class="durationClass">{{duration}}</span></td>
http://git-wip-us.apache.org/repos/asf/spark/blob/9e86e6ef/core/src/main/resources/org/apache/spark/ui/static/historypage.js
----------------------------------------------------------------------
diff --git a/core/src/main/resources/org/apache/spark/ui/static/historypage.js b/core/src/main/resources/org/apache/spark/ui/static/historypage.js
index 167c802..4ff0831 100644
--- a/core/src/main/resources/org/apache/spark/ui/static/historypage.js
+++ b/core/src/main/resources/org/apache/spark/ui/static/historypage.js
@@ -123,13 +123,28 @@ $(document).ready(function() {
if (app["attempts"].length > 1) {
hasMultipleAttempts = true;
}
- var num = app["attempts"].length;
+
+ var maxAttemptId = null
for (j in app["attempts"]) {
var attempt = app["attempts"][j];
+ if (attempt['attemptId'] != null) {
+ if (maxAttemptId == null || attempt['attemptId'] > maxAttemptId) {
+ maxAttemptId = attempt['attemptId']
+ }
+ }
+
attempt["startTime"] = formatDate(attempt["startTime"]);
attempt["endTime"] = formatDate(attempt["endTime"]);
attempt["lastUpdated"] = formatDate(attempt["lastUpdated"]);
- var app_clone = {"id" : id, "name" : name, "num" : num, "attempts" : [attempt]};
+
+ var url = null
+ if (maxAttemptId == null) {
+ url = "/history/" + id + "/"
+ } else {
+ url = "/history/" + id + "/" + maxAttemptId + "/"
+ }
+
+ var app_clone = {"id" : id, "name" : name, "url" : url, "attempts" : [attempt]};
array.push(app_clone);
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org