You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by zhen <z....@latrobe.edu.au> on 2014/06/14 03:21:48 UTC
spark master UI does not keep detailed application history
I have been trying to get detailed history of previous spark shell executions
(after exiting spark shell). In standalone mode and Spark 1.0, I think the
spark master UI is supposed to provide detailed execution statistics of all
previously run jobs. This is supposed to be viewable by clicking on the
"Spark shell" link for each job. But when I click link "Spark shell" nothing
happens. Am I missing something?
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/spark-master-UI-does-not-keep-detailed-application-history-tp7608.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: spark master UI does not keep detailed application history
Posted by Andrew Or <an...@databricks.com>.
Are you referring to accessing a SparkUI for an application that has
finished? First you need to enable event logging while the application is
still running. In Spark 1.0, you set this by adding a line to
$SPARK_HOME/conf/spark-defaults.conf:
spark.eventLog.enabled true
Other than that, the content served on the master UI is largely the same as
before 1.0 is introduced.
2014-06-14 16:43 GMT-07:00 wxhsdp <wx...@gmail.com>:
> hi, zhen
> i met the same problem in ec2, application details can not be accessed.
> but i can read stdout
> and stderr. the problem has not been solved yet
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/spark-master-UI-does-not-keep-detailed-application-history-tp7608p7635.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
Re: spark master UI does not keep detailed application history
Posted by wxhsdp <wx...@gmail.com>.
hi, zhen
i met the same problem in ec2, application details can not be accessed.
but i can read stdout
and stderr. the problem has not been solved yet
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/spark-master-UI-does-not-keep-detailed-application-history-tp7608p7635.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.