[jira] [Created] (SPARK-27053) How about allowing engineers to use a different ExecutorBackend in StandAlone mode?

["Ross Brigoli (JIRA)" <ji...@apache.org>]

Ross Brigoli created SPARK-27053:
------------------------------------

             Summary: How about allowing engineers to use a different ExecutorBackend in StandAlone mode?
                 Key: SPARK-27053
                 URL: https://issues.apache.org/jira/browse/SPARK-27053
             Project: Spark
          Issue Type: Improvement
          Components: Deploy, Spark Submit
    Affects Versions: 2.3.3
            Reporter: Ross Brigoli


In Standalone mode, the command for starting an Executor JVM in is hardcoded to use org.apache.spark.executor.CoarseGrainedExecutorBackend. There seems to be no way to configure the submit operation to use a custom ExecutorBackend (a subclass of CoarseGrainedExecutorBackend).

This is very useful when engineers need to initialize things like starting a JDBC connection and Closing JDBC connection once per Executor.

At line 103 of StandaloneSchedulerBackend.scala, why not make the fully qualified name of the executor backend class configurable? And then fall back to this default executor backend class if it's not configured.

{{val command = Command("{color:#FF0000}*org.apache.spark.executor.CoarseGrainedExecutorBackend*{color}",}}
{{      args, sc.executorEnvs, classPathEntries ++ testingClassPath, libraryPathEntries, javaOpts)}}

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org