[jira] [Created] (SPARK-15703) Spark UI doesn't show all tasks as completed when they are

["Thomas Graves (JIRA)" <ji...@apache.org>]

Thomas Graves created SPARK-15703:
-------------------------------------

             Summary: Spark UI doesn't show all tasks as completed when they are
                 Key: SPARK-15703
                 URL: https://issues.apache.org/jira/browse/SPARK-15703
             Project: Spark
          Issue Type: Bug
          Components: Web UI
    Affects Versions: 2.0.0
            Reporter: Thomas Graves
            Priority: Critical


The Spark UI doesn't seem to be showing all the tasks and metrics.

I ran a job with 100000 tasks but Detail stage page says it completed 93029:

Summary Metrics for 93029 Completed Tasks

The Stages for all jobs pages list that only 89519/100000 tasks finished but its completed.  The metrics for shuffled write and input are also incorrect.

I will attach screen shots.

I checked the logs and it does show that all the tasks actually finished.

16/06/01 16:15:42 INFO TaskSetManager: Finished task 59880.0 in stage 2.0 (TID 54038) in 265309 ms on 10.213.45.51 (100000/100000)
16/06/01 16:15:42 INFO YarnClusterScheduler: Removed TaskSet 2.0, whose tasks have all completed, from pool



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org