You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "David McWhorter (Jira)" <ji...@apache.org> on 2019/10/08 20:54:00 UTC
[jira] [Commented] (SPARK-22256) Introduce
spark.mesos.driver.memoryOverhead
[ https://issues.apache.org/jira/browse/SPARK-22256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16947177#comment-16947177 ]
David McWhorter commented on SPARK-22256:
-----------------------------------------
I created this pull request https://github.com/pmackles/spark/pull/1 to update the branch for this pull request with changes on branch https://github.com/dmcwhorter/spark/tree/dmcwhorter-SPARK-22256.
The updates do the following:
1. Rebase pmackles/paul-SPARK-22256 onto the latest apache/spark master branch
2. Add a test case for when the default value of spark.mesos.driver.memoryOverhead is 10% of driver memory as requested in apache#21006
My hope is this update will make this pull request ready to merge and include in the next spark release.
> Introduce spark.mesos.driver.memoryOverhead
> --------------------------------------------
>
> Key: SPARK-22256
> URL: https://issues.apache.org/jira/browse/SPARK-22256
> Project: Spark
> Issue Type: Improvement
> Components: Mesos
> Affects Versions: 2.2.0, 2.3.0
> Reporter: Cosmin Lehene
> Priority: Minor
> Labels: docker, memory, mesos
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> When running spark driver in a container such as when using the Mesos dispatcher service, we need to apply the same rules as for executors in order to avoid the JVM going over the allotted limit and then killed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org