You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "shahid (JIRA)" <ji...@apache.org> on 2019/04/17 02:08:00 UTC
[jira] [Comment Edited] (SPARK-16859) History Server storage
information is missing
[ https://issues.apache.org/jira/browse/SPARK-16859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819662#comment-16819662 ]
shahid edited comment on SPARK-16859 at 4/17/19 2:07 AM:
---------------------------------------------------------
[~Hauer] [~toopt4] Can you try enabling "spark.eventLog.logBlockUpdates.enabled=true" and see, if still History server storage tab is empty?
was (Author: shahid):
[~Hauer][~toopt4] Can you try enabling "spark.eventLog.logBlockUpdates.enabled=true" and see, if still History server storage tab is empty?
> History Server storage information is missing
> ---------------------------------------------
>
> Key: SPARK-16859
> URL: https://issues.apache.org/jira/browse/SPARK-16859
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.6.2, 2.0.0
> Reporter: Andrei Ivanov
> Priority: Major
> Labels: historyserver, newbie
>
> It looks like job history storage tab in history server is broken for completed jobs since *1.6.2*.
> More specifically it's broken since [SPARK-13845|https://issues.apache.org/jira/browse/SPARK-13845].
> I've fixed for my installation by effectively reverting the above patch ([see|https://github.com/EinsamHauer/spark/commit/3af62ea09af8bb350c8c8a9117149c09b8feba08]).
> IMHO, the most straightforward fix would be to implement _SparkListenerBlockUpdated_ serialization to JSON in _JsonProtocol_ making sure it works from _ReplayListenerBus_.
> The downside will be that it will still work incorrectly with pre patch job histories. But then, it doesn't work since *1.6.2* anyhow.
> PS: I'd really love to have this fixed eventually. But I'm pretty new to Apache Spark and missing hands on Scala experience. So I'd prefer that it be fixed by someone experienced with roadmap vision. If nobody volunteers I'll try to patch myself.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org