You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "jiaan.geng (JIRA)" <ji...@apache.org> on 2019/01/17 04:21:00 UTC

[jira] [Created] (SPARK-26641) Seperate capacity Configurations of different event queue.

jiaan.geng created SPARK-26641:
----------------------------------

             Summary: Seperate capacity Configurations of different event queue.
                 Key: SPARK-26641
                 URL: https://issues.apache.org/jira/browse/SPARK-26641
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
    Affects Versions: 2.4.0, 2.3.0, 3.0.0
            Reporter: jiaan.geng


I always maintance a spark on yarn cluster on line.I always found the error:

`Dropping event from queue eventLog. This likely means one of the listeners is too slow and cannot keep up with the rate at which tasks are being started by the scheduler.`

Spark event log write to a hdfs cluster on line, and this hdfs cluster support write a few TB.The performance issuse of write frequently, lead to the `EventLoggingListener` exists the bottleneck.

But other event queue appear the problem rarely,so I think seperate the configurations of different event queue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org