[jira] [Issue Comment Deleted] (SPARK-16382) YARN - Dynamic allocation with spark.executor.instances should increase max executors.

["Sean Owen (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-16382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Owen updated SPARK-16382:
------------------------------
    Comment: was deleted

(was: User 'srowen' has created a pull request for this issue:
https://github.com/apache/spark/pull/14826)

> YARN - Dynamic allocation with spark.executor.instances should increase max executors.
> --------------------------------------------------------------------------------------
>
>                 Key: SPARK-16382
>                 URL: https://issues.apache.org/jira/browse/SPARK-16382
>             Project: Spark
>          Issue Type: Bug
>          Components: YARN
>            Reporter: Ryan Blue
>
> SPARK-13723 changed the behavior of dynamic allocation when {{--num-executors}} ({{spark.executor.instances}}) is set. Rather than turning off dynamic allocation, the value is used as the initial number of executors. This did not change the behavior of {{spark.dynamicAllocation.maxExecutors}}. We've noticed that some users set {{--num-executors}} higher than the max and the expectation is that the max increases.
> I think that either max should be increased, or Spark should fail and complain that the executors requested is higher than the max.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org