You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (Jira)" <ji...@apache.org> on 2019/10/08 05:42:15 UTC
[jira] [Resolved] (SPARK-21084) Improvements to dynamic allocation
for notebook use cases
[ https://issues.apache.org/jira/browse/SPARK-21084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-21084.
----------------------------------
Resolution: Incomplete
> Improvements to dynamic allocation for notebook use cases
> ---------------------------------------------------------
>
> Key: SPARK-21084
> URL: https://issues.apache.org/jira/browse/SPARK-21084
> Project: Spark
> Issue Type: Umbrella
> Components: Block Manager, Scheduler, Spark Core, YARN
> Affects Versions: 2.2.0, 2.3.0
> Reporter: Frederick Reiss
> Priority: Major
> Labels: bulk-closed
>
> One important application of Spark is to support many notebook users with a single YARN or Spark Standalone cluster. We at IBM have seen this requirement across multiple deployments of Spark: on-premises and private cloud deployments at our clients, as well as on the IBM cloud. The scenario goes something like this: "Every morning at 9am, 500 analysts log into their computers and start running Spark notebooks intermittently for the next 8 hours." I'm sure that many other members of the community are interested in making similar scenarios work.
>
> Dynamic allocation is supposed to support these kinds of use cases by shifting cluster resources towards users who are currently executing scalable code. In our own testing, we have encountered a number of issues with using the current implementation of dynamic allocation for this purpose:
> *Issue #1: Starvation.* A Spark job acquires all available containers, preventing other jobs or applications from starting.
> *Issue #2: Request latency.* Jobs that would normally finish in less than 30 seconds take 2-4x longer than normal with dynamic allocation.
> *Issue #3: Unfair resource allocation due to cached data.* Applications that have cached RDD partitions hold onto executors indefinitely, denying those resources to other applications.
> *Issue #4: Loss of cached data leads to thrashing.* Applications repeatedly lose partitions of cached RDDs because the underlying executors are removed; the applications then need to rerun expensive computations.
>
> This umbrella JIRA covers efforts to address these issues by making enhancements to Spark.
> Here's a high-level summary of the current planned work:
> * [SPARK-21097]: Preserve an executor's cached data when shutting down the executor.
> * [SPARK-21122]: Make Spark give up executors in a controlled fashion when the RM indicates it is running low on capacity.
> * (JIRA TBD): Reduce the delay for dynamic allocation to spin up new executors.
> Note that this overall plan is subject to change, and other members of the community should feel free to suggest changes and to help out.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org