You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Liang-Chi Hsieh (JIRA)" <ji...@apache.org> on 2016/04/10 11:51:25 UTC
[jira] [Updated] (SPARK-9882) Priority-based scheduling for Spark
applications
[ https://issues.apache.org/jira/browse/SPARK-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Liang-Chi Hsieh updated SPARK-9882:
-----------------------------------
Description:
We implement this patch because in our daily usage of Spark we found that applications scheduling is an important issue for utilizing cluster resources. Currently in standalone mode, we don't have efficient way to schedule multiple Spark applications on a single cluster. We need an efficient way to manage different Spark applications with different priorities.
was:
This is a priority-based scheduling based on the interface proposed in #7958. It uses a XML configuration file to specify the submission pools for applications. This patch adds a new parameter `--pool` to SparkSubmit under standalone mode to specify which pool the submitted application should be assigned to. The priority of the submitted application is defined by the assigned pool. It also defines the cores it can acquire.
> Priority-based scheduling for Spark applications
> ------------------------------------------------
>
> Key: SPARK-9882
> URL: https://issues.apache.org/jira/browse/SPARK-9882
> Project: Spark
> Issue Type: New Feature
> Components: Spark Core
> Reporter: Liang-Chi Hsieh
>
> We implement this patch because in our daily usage of Spark we found that applications scheduling is an important issue for utilizing cluster resources. Currently in standalone mode, we don't have efficient way to schedule multiple Spark applications on a single cluster. We need an efficient way to manage different Spark applications with different priorities.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org