You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean Owen (JIRA)" <ji...@apache.org> on 2016/01/02 15:22:39 UTC
[jira] [Resolved] (SPARK-6197) handle json parse exception for
eventlog file not finished writing
[ https://issues.apache.org/jira/browse/SPARK-6197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved SPARK-6197.
------------------------------
Resolution: Fixed
Fix Version/s: 1.4.0
Target Version/s: (was: 1.3.2, 1.4.2)
> handle json parse exception for eventlog file not finished writing
> -------------------------------------------------------------------
>
> Key: SPARK-6197
> URL: https://issues.apache.org/jira/browse/SPARK-6197
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 1.3.0
> Reporter: Zhang, Liye
> Assignee: Zhang, Liye
> Priority: Minor
> Fix For: 1.4.0
>
>
> This is a following JIRA for [SPARK-6107|https://issues.apache.org/jira/browse/SPARK-6107]. In [SPARK-6107|https://issues.apache.org/jira/browse/SPARK-6107], webUI can display event log files that with suffix *.inprogress*. However, the eventlog file may be not finished writing for some abnormal cases (e.g. Ctrl+C), In which case, the file maybe truncated in the last line, leading to the line being not in valid Json format. Which will cause Json parse exception when reading the file.
> For this case, we can just ignore the last line content, since the history for abnormal cases showed on web is only a reference for user, it can demonstrate the past status of the app before terminated abnormally (we can not guarantee the history can show exactly the last moment when app encounter the abnormal situation).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org