You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Josh Rosen (JIRA)" <ji...@apache.org> on 2015/01/26 23:07:35 UTC

[jira] [Created] (SPARK-5411) Allow SparkListeners to be specified in SparkConf and loaded when creating SparkContext

Josh Rosen created SPARK-5411:
---------------------------------

             Summary: Allow SparkListeners to be specified in SparkConf and loaded when creating SparkContext
                 Key: SPARK-5411
                 URL: https://issues.apache.org/jira/browse/SPARK-5411
             Project: Spark
          Issue Type: New Feature
          Components: Spark Core
            Reporter: Josh Rosen
            Assignee: Josh Rosen


It would be nice if there was a mechanism to allow SparkListeners to be registered through SparkConf settings.  This would allow monitoring frameworks to be easily injected into Spark programs without having to modify those programs' code.

I propose to introduce a new configuration option, {{spark.extraListeners}}, that allows SparkListeners to be specified in SparkConf and registered before the SparkContext is created.  Here is the proposed documentation for the new option:

{quote}
A comma-separated list of classes that implement SparkListener; when initializing SparkContext, instances of these classes will be created and registered with Spark's listener bus. If a class has a single-argument constructor that accepts a SparkConf, that constructor will be called; otherwise, a zero-argument constructor will be called. If no valid constructor can be found, the SparkContext creation will fail with an exception.
{quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org