You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (Jira)" <ji...@apache.org> on 2021/04/28 10:13:00 UTC
[jira] [Assigned] (SPARK-35229) Spark Job web page is extremely
slow while there are more than 1500 events in timeline
[ https://issues.apache.org/jira/browse/SPARK-35229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-35229:
------------------------------------
Assignee: Apache Spark
> Spark Job web page is extremely slow while there are more than 1500 events in timeline
> --------------------------------------------------------------------------------------
>
> Key: SPARK-35229
> URL: https://issues.apache.org/jira/browse/SPARK-35229
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 3.1.1
> Reporter: Wang Yuan
> Assignee: Apache Spark
> Priority: Blocker
>
> In a spark streaming application, there are 1000+ executors, more than 2000 events (executors events, job events) generated, then the jobs/job web page of spark is not able to show. The browser (chrome, firefox, safari) freezes. I had to open another window to open stages, executors pages by their addresses manually.
> The jobs page is the home page, so this is rather annoyed. The problem is that the vis-timeline rending is too slow.
> Their are some suggestion:
> 1) the page should not render timeline when the page is loading unless user clicks the link.
> 2) the executor group and job group should be separated, and user can choose to show one, both or neither.
> 3) the executor group should display executor event by time horizontally. Currently it is displayed executors one line by one line. If more than 100 executors, the page is not that good.
> 4)the vis-timeline library is not maintained anymore since 2017. Should be replaced a new one like [https://github.com/visjs/vis-timeline]
> 5) it is also good to get recent events, e.g. 500, and load more if user want see more, however the data could all be loaded once.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org