You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean Owen (JIRA)" <ji...@apache.org> on 2015/04/13 13:14:12 UTC
[jira] [Resolved] (SPARK-6868) Container link broken on Spark UI
Executors page when YARN is set to HTTPS_ONLY
[ https://issues.apache.org/jira/browse/SPARK-6868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved SPARK-6868.
------------------------------
Resolution: Fixed
Fix Version/s: 1.4.0
1.3.2
Assignee: Dean Chen
> Container link broken on Spark UI Executors page when YARN is set to HTTPS_ONLY
> -------------------------------------------------------------------------------
>
> Key: SPARK-6868
> URL: https://issues.apache.org/jira/browse/SPARK-6868
> Project: Spark
> Issue Type: Bug
> Components: YARN
> Affects Versions: 1.1.0, 1.1.1, 1.2.0, 1.2.1, 1.3.0
> Reporter: Dean Chen
> Assignee: Dean Chen
> Fix For: 1.3.2, 1.4.0
>
> Attachments: Screen Shot 2015-04-11 at 11.49.21 PM.png
>
>
> The stdout and stderr log links on the executor page will use the http:// prefix even if the node manager does not support http and only https via setting yarn.http.policy=HTTPS_ONLY.
> Unfortunately the unencrypted http link in that case does not return a 404 but a binary file containing random binary chars. This causes a lot of confusion for the end user since it seems like the log file exists and is just filled with garbage. (see attached screenshot)
> The fix is to prefix container log links with https:// instead of http:// if yarn.http.policy=HTTPS_ONLY. YARN's job page has this exact logic as seen here: https://github.com/apache/hadoop/blob/e1109fb65608a668cd53dc324dadc6f63a74eeb9/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/JobBlock.java#L108
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org