You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Thomas Graves (JIRA)" <ji...@apache.org> on 2014/08/22 23:50:12 UTC
[jira] [Resolved] (SPARK-1287) yarn alpha and stable Client
calculateAMMemory routines are different
[ https://issues.apache.org/jira/browse/SPARK-1287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Graves resolved SPARK-1287.
----------------------------------
Resolution: Duplicate
duplicate of SPARK-2140
> yarn alpha and stable Client calculateAMMemory routines are different
> ---------------------------------------------------------------------
>
> Key: SPARK-1287
> URL: https://issues.apache.org/jira/browse/SPARK-1287
> Project: Spark
> Issue Type: Bug
> Components: YARN
> Affects Versions: 0.9.0, 1.0.0
> Reporter: Thomas Graves
>
> The yarn alpha version of calculateAMMemory takes into account the minimum resource capability and also subtracts out YarnAllocationHandler.MEMORY_OVERHEAD.
> The yarn stable version just sets the -Xmx to whatever is passed in by the user.
> The 2 of these should be the same.
>
> Personally I also think its weird how spark currently takes whatever user passes in for memory and adds YarnAllocationHandler.MEMORY_OVERHEAD to request to RM. This can be confusing to users. We should revisit all of this and commonize the stable/alpha code where possible.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org