[jira] [Updated] (SPARK-12384) Allow -Xms to be set differently then -Xmx

["Sean Owen (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-12384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Owen updated SPARK-12384:
------------------------------
    Priority: Minor  (was: Major)

I tend to think it's not useful to set -Xms to anything non-trivially large. The extra GCs and growing the heap doesn't matter that much. -Xms doesn't actually make the JVM consume that much memory at startup (right?), it just makes the JVM feel free to use memory instead of trying to GC, up to that large limit. The net effect may be using more memory than necessary, indeed.

What about removing setting -Xms entirely?

> Allow -Xms to be set differently then -Xmx
> ------------------------------------------
>
>                 Key: SPARK-12384
>                 URL: https://issues.apache.org/jira/browse/SPARK-12384
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Submit, YARN
>    Affects Versions: 1.6.0
>            Reporter: Thomas Graves
>            Priority: Minor
>
> Currently Spark automatically sets the -Xms parameter to be the same as the -Xmx parameter. We should allow the user to set this separately.
> The main use case here is if I'm running the spark-shell on a shared gateway. Many users specify a larger memory size then needed and will never use that much memory, so all its doing is preventing other users from potentially using that memory.  Allowing it to be less is just more multi-tenant friendly.
> I think it makes sense to leave this for cluster mode, although if a user really wants to override I don't see why we shouldn't let them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org