[jira] [Created] (SPARK-27394) The staleness of UI may last minutes or hours when no tasks start or finish

["Shixiong Zhu (JIRA)" <ji...@apache.org>]

Shixiong Zhu created SPARK-27394:
------------------------------------

             Summary: The staleness of UI may last minutes or hours when no tasks start or finish
                 Key: SPARK-27394
                 URL: https://issues.apache.org/jira/browse/SPARK-27394
             Project: Spark
          Issue Type: Bug
          Components: Web UI
    Affects Versions: 2.4.1, 2.4.0
            Reporter: Shixiong Zhu


Run the following codes on a cluster that has at least 2 cores.
{code}
sc.makeRDD(1 to 1000, 1000).foreach { i =>
  Thread.sleep(300000)
}
{code}

The jobs page will just show one running task.

This is because when the second task event calls "AppStatusListener.maybeUpdate" for a job, it will just ignore since the gap between two events is smaller than `spark.ui.liveUpdate.period`.

After the second task event, in the above case, because there won't be any other task events, the Spark UI will be always stale until the next task event gets fired (after 300 seconds).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org