You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2016/01/26 15:33:40 UTC
[jira] [Commented] (SPARK-7889) Jobs progress of apps on complete
page of HistoryServer shows uncompleted
[ https://issues.apache.org/jira/browse/SPARK-7889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15117288#comment-15117288 ]
Steve Loughran commented on SPARK-7889:
---------------------------------------
I'm going to note something something problematic about probing HDFS for changes, HDFS-5478 reports that flushing fs data doesn't immediately update fs length. So detecting changes in log files saved to HDFS isn't likely to be that immediate. the SPARK-1537 timeline binding won't have this problem, nor will logs saved to other filesystems (object stores have different issues)
> Jobs progress of apps on complete page of HistoryServer shows uncompleted
> -------------------------------------------------------------------------
>
> Key: SPARK-7889
> URL: https://issues.apache.org/jira/browse/SPARK-7889
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Reporter: meiyoula
> Priority: Minor
>
> When running a SparkPi with 2000 tasks, cliking into the app on incomplete page, the job progress shows 400/2000. After the app is completed, the app goes to complete page from incomplete, and now cliking into the app, the job progress still shows 400/2000.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org