You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Jean-Baptiste Onofré (JIRA)" <ji...@apache.org> on 2015/10/05 20:15:27 UTC
[jira] [Updated] (SPARK-10927) Spark history uses the application
name instead of the ID
[ https://issues.apache.org/jira/browse/SPARK-10927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Baptiste Onofré updated SPARK-10927:
-----------------------------------------
Description:
Setting spark.eventLog.enabled to true, and a folder location for spark.eventLog.dir provides the history UI for completed jobs.
It works fine for jobs without arguments, but if the job expects some arguments (like JavaWordCount which expects the source file location), the UI is not possible to provide application details:
{code}
Application history not found (app-20151005185136-0002)
No event logs found for application JavaWordCount in file:/tmp/spark. Did you specify the correct logging directory?
{code}
However, in /tmp/spark, the file app-20151005185136-0002 is there. It seems that the UI uses the application name (JavaWordCount) instead of the application ID (app-20151005185136-0002) to get history details.
I will work on a fix around that.
was:
Setting spark.eventLog.enabled to true, and a folder location for spark.eventLog.dir provides the history UI for completed jobs.
It works fine for jobs without arguments, but if the job expects some arguments (like JavaWordCount which expects the source file location), the UI is not possible to provide application details:
{code}
Application history not found (app-20151005185136-0002)
No event logs found for application JavaWordCount in file:/tmp/spark. Did you specify the correct logging directory?
{/code}
However, in /tmp/spark, the file app-20151005185136-0002 is there. It seems that the UI uses the application name (JavaWordCount) instead of the application ID (app-20151005185136-0002) to get history details.
I will work on a fix around that.
> Spark history uses the application name instead of the ID
> ---------------------------------------------------------
>
> Key: SPARK-10927
> URL: https://issues.apache.org/jira/browse/SPARK-10927
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 1.5.1
> Reporter: Jean-Baptiste Onofré
>
> Setting spark.eventLog.enabled to true, and a folder location for spark.eventLog.dir provides the history UI for completed jobs.
> It works fine for jobs without arguments, but if the job expects some arguments (like JavaWordCount which expects the source file location), the UI is not possible to provide application details:
> {code}
> Application history not found (app-20151005185136-0002)
> No event logs found for application JavaWordCount in file:/tmp/spark. Did you specify the correct logging directory?
> {code}
> However, in /tmp/spark, the file app-20151005185136-0002 is there. It seems that the UI uses the application name (JavaWordCount) instead of the application ID (app-20151005185136-0002) to get history details.
> I will work on a fix around that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org