[jira] [Created] (SPARK-26192) MesosClusterScheduler reads options from dispatcher conf instead of submission conf

["Martin Loncaric (JIRA)" <ji...@apache.org>]

Martin Loncaric created SPARK-26192:
---------------------------------------

             Summary: MesosClusterScheduler reads options from dispatcher conf instead of submission conf
                 Key: SPARK-26192
                 URL: https://issues.apache.org/jira/browse/SPARK-26192
             Project: Spark
          Issue Type: Bug
          Components: Mesos
    Affects Versions: 2.4.0, 2.3.2, 2.3.1, 2.3.0
            Reporter: Martin Loncaric


There are at least two options accessed in MesosClusterScheduler that should come from the submission's configuration instead of the dispatcher's:

spark.app.name
spark.mesos.fetchCache.enable

This means that all Mesos tasks for Spark drivers have uninformative names of the form "Driver for (MainClass)" rather than the configured application name, and Spark drivers never cache files. Coincidentally, the spark.mesos.fetchCache.enable option is misnamed, as referenced in the linked JIRA.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org