[jira] [Assigned] (SPARK-33215) Speed up event log download by skipping UI rebuild

["Apache Spark (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-33215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Apache Spark reassigned SPARK-33215:
------------------------------------

    Assignee:     (was: Apache Spark)

> Speed up event log download by skipping UI rebuild
> --------------------------------------------------
>
>                 Key: SPARK-33215
>                 URL: https://issues.apache.org/jira/browse/SPARK-33215
>             Project: Spark
>          Issue Type: Improvement
>          Components: Web UI
>    Affects Versions: 2.4.7, 3.0.1
>            Reporter: Baohe Zhang
>            Priority: Major
>
> Right now, when we want to download the event logs from the spark history server(SHS), SHS will need to parse entire the event log to rebuild UI, and this is just for view permission checks. UI rebuilding is a time-consuming and memory-intensive task, especially for large logs. However, this process is unnecessary for event log download.
> This patch enables SHS to check UI view permissions of a given app/attempt for a given user, without rebuilding the UI. This is achieved by adding a method "checkUIViewPermissions(appId: String, attemptId: Option[String], user: String): Boolean" to many layers of history server components.
> With this patch, UI rebuild can be skipped when downloading event logs from the history server. Thus the time of downloading a GB scale event log can be reduced from several minutes to several seconds, and the memory consumption of UI rebuilding can be avoided.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org