You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "jiaan.geng (JIRA)" <ji...@apache.org> on 2019/01/17 09:47:00 UTC
[jira] [Resolved] (SPARK-26641) Seperate capacity Configurations of
different event queue.
[ https://issues.apache.org/jira/browse/SPARK-26641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
jiaan.geng resolved SPARK-26641.
--------------------------------
Resolution: Won't Fix
> Seperate capacity Configurations of different event queue.
> ----------------------------------------------------------
>
> Key: SPARK-26641
> URL: https://issues.apache.org/jira/browse/SPARK-26641
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 2.3.0, 2.4.0, 3.0.0
> Reporter: jiaan.geng
> Priority: Minor
>
> I always maintance a spark on yarn cluster on line.I always found the error:
> `Dropping event from queue eventLog. This likely means one of the listeners is too slow and cannot keep up with the rate at which tasks are being started by the scheduler.`
> Spark event log write to a hdfs cluster on line, and this hdfs cluster support write a few TB every day.The performance issuse of write frequently, lead to the `EventLoggingListener` exists the bottleneck.
> But other event queue appear the problem rarely,so I think seperate the configurations of different event queue.
> There add error log on line:
>
> {code:java}
> 18/12/14 19:02:15 ERROR AsyncEventQueue: Dropping event from queue eventLog. This likely means one of the listeners is too slow and cannot keep up with the rate at which tasks are being started by the scheduler.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org