[jira] [Created] (SPARK-35229) Spark Job web page is extremely slow while there are more than 1500 events in timeline

["Wang Yuan (Jira)" <ji...@apache.org>]

Wang Yuan created SPARK-35229:
---------------------------------

             Summary: Spark Job web page is extremely slow while there are more than 1500 events in timeline
                 Key: SPARK-35229
                 URL: https://issues.apache.org/jira/browse/SPARK-35229
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
    Affects Versions: 3.1.1
            Reporter: Wang Yuan


In a spark streaming application, there are 1000+ executors, more than 2000 events (executors events, job events) generated, then the jobs/job web page of spark is not able to show. The browser (chrome, firefox, safari) freezes. I had to open another window to open stages, executors pages by their addresses manually.

The jobs page is the home page, so this is rather annoyed. The problem is that the vis-timeline rending is too slow.

Their are some suggestion:

1) the page should not render timeline when the page is loading unless user clicks the link.

2) the executor group and job group should be separated, and user can choose to show one, both or neither.

3) the executor group should display executor event by time horizontally. Currently it is displayed executors one line by one line. If more than 100 executors, the page is not that good.

4)the vis-timeline library is not maintained anymore since 2017. Should be replaced a new one like [https://github.com/visjs/vis-timeline]

5) it is also good to get recent events, e.g. 500, and load more if user want see more, however the data could all be loaded once.

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org