You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "duyanghao (JIRA)" <ji...@apache.org> on 2017/07/09 02:36:00 UTC
[jira] [Comment Edited] (SPARK-21348) Spark history Server load too
slow when eventlog is large
[ https://issues.apache.org/jira/browse/SPARK-21348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16079421#comment-16079421 ]
duyanghao edited comment on SPARK-21348 at 7/9/17 2:35 AM:
-----------------------------------------------------------
[~jerryshao] ok, i will follow [SPARK-18085](https://issues.apache.org/jira/browse/SPARK-18085) to trace this issue.
was (Author: duyanghao):
@Saisai Shao ok, i will follow [SPARK-18085](https://issues.apache.org/jira/browse/SPARK-18085) to trace this issue.
> Spark history Server load too slow when eventlog is large
> ---------------------------------------------------------
>
> Key: SPARK-21348
> URL: https://issues.apache.org/jira/browse/SPARK-21348
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 2.1.0
> Reporter: duyanghao
> Priority: Critical
>
> There is a eventlog file in hdfs as below:
> ```
> -rwxrwx--- 3 root supergroup 9634238709 2017-07-08 04:37 hdfs://xxxxx/shared/spark-logs/xxxxx/xxxx.lz4
> ```
> and i start a spark history server to loading that eventlog file,but it is too slow to get spark web ui through that spark history server.
> it has requested for several hours but nothing about spark application web ui appears in the browser
> And the cpu usage of spark history server is 105.3%
> total used-memory in host is 9G
> The host has 48 Cores+130G Memory,enough Computing resources for spark history server.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org