[jira] [Created] (SPARK-5095) Support launching multiple mesos executors in coarse grained mesos mode

["Timothy Chen (JIRA)" <ji...@apache.org>]

Timothy Chen created SPARK-5095:
-----------------------------------

             Summary: Support launching multiple mesos executors in coarse grained mesos mode
                 Key: SPARK-5095
                 URL: https://issues.apache.org/jira/browse/SPARK-5095
             Project: Spark
          Issue Type: Improvement
            Reporter: Timothy Chen


Currently in coarse grained mesos mode, it's expected that we only launch one Mesos executor that launches one JVM process to launch multiple spark executors.

However, this become a problem when the JVM process launched is larger than an ideal size (30gb is recommended value from databricks), which causes GC problems reported on the mailing list.

We should support launching mulitple executors when large enough resources are available for spark to use, and these resources are still under the configured limit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org