You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "SuYan (JIRA)" <ji...@apache.org> on 2016/04/21 10:38:25 UTC
[jira] [Comment Edited] (SPARK-14750) Make historyServer refer
application log in hdfs
[ https://issues.apache.org/jira/browse/SPARK-14750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15251558#comment-15251558 ]
SuYan edited comment on SPARK-14750 at 4/21/16 8:37 AM:
--------------------------------------------------------
historyServer for spark on yarn, the logUrl was something like:
http://yarn-nodemanager-host:port/node/containerlogs/container_1447144693824_768936_01_000021/h_mifi/spark.log?start=-4096
user can refer spark.log for running Application through ExecutorsPage, but after application was finished, stdout,stderr,gc.log will moved from nodeManager folder to yarn hdfs folder.
So user can't use
http://yarn-nodemanager-host:port/node/containerlogs/container_1447144693824_768936_01_000021/h_mifi/spark.log?start=-4096 to get application logs, because it was removed.
and I not familiar with spark standalone mode, will user can refer stdout/stderr/gclog after application was finished?
it can be more convenient if user can refer application log in historyServer, right?
was (Author: suyan):
historyServer for spark on yarn, the logUrl was something like:
http://yarn-nodemanager-host:port/node/containerlogs/container_1447144693824_768936_01_000021/h_mifi/spark.log?start=-4096
user can refer spark.log for running Application through ExecutorsPage, but after application was finished, stdout,stderr,gc.log will moved from nodeManager folder to yarn hdfs folder.
So user can't use
http://yarn-nodemanager-host:port/node/containerlogs/container_1447144693824_768936_01_000021/h_mifi/spark.log?start=-4096 to get application logs, because it was removed.
> Make historyServer refer application log in hdfs
> ------------------------------------------------
>
> Key: SPARK-14750
> URL: https://issues.apache.org/jira/browse/SPARK-14750
> Project: Spark
> Issue Type: Improvement
> Components: Web UI
> Affects Versions: 1.6.1
> Reporter: SuYan
>
> Make history server refer application log, just like MR history server
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org