You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Ryan Williams (JIRA)" <ji...@apache.org> on 2014/12/01 06:46:13 UTC
[jira] [Created] (SPARK-4665) Config value for setting yarn
container overhead to a fraction of executor memory
Ryan Williams created SPARK-4665:
------------------------------------
Summary: Config value for setting yarn container overhead to a fraction of executor memory
Key: SPARK-4665
URL: https://issues.apache.org/jira/browse/SPARK-4665
Project: Spark
Issue Type: Improvement
Components: YARN
Reporter: Ryan Williams
Currently, the {{spark.yarn.executor.memoryOverhead}} config lets you specify a number of MB of "overhead" memory to allocate for each executor.
It turns out that it is more useful to set this to a specific fraction of the executor memory size; currently a lower-bound of 7% is the default.
This lower-bound percentage should be configurable, as well; I've heard rules of thumb that closer to 10% is desirable, and in general it makes more sense to be able to specify the fraction you're targeting than to specify an absolute number of MB.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org