You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (JIRA)" <ji...@apache.org> on 2019/06/09 20:50:00 UTC
[jira] [Commented] (SPARK-27719) Set maxDisplayLogSize for spark
history server
[ https://issues.apache.org/jira/browse/SPARK-27719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16859558#comment-16859558 ]
Dongjoon Hyun commented on SPARK-27719:
---------------------------------------
Hi, [~hao.li]. Do you have big log files over 200GB?
In reality, the logs grows. In other words, it started with small sizes and will grow and exceed the limit.
Are you expecting some entries disappear when you refresh Spark History Server page?
cc [~vanzin]. How do you think about this suggestion?
> Set maxDisplayLogSize for spark history server
> ----------------------------------------------
>
> Key: SPARK-27719
> URL: https://issues.apache.org/jira/browse/SPARK-27719
> Project: Spark
> Issue Type: Improvement
> Components: Web UI
> Affects Versions: 3.0.0
> Reporter: hao.li
> Priority: Minor
>
> Sometimes a very large eventllog may be useless, and parses it may waste many resources.
> It may be useful to avoid parse large enventlogs by setting a configuration spark.history.fs.maxDisplayLogSize.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org