You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Yinan Li (JIRA)" <ji...@apache.org> on 2018/05/10 18:42:00 UTC
[jira] [Updated] (SPARK-24137) [K8s] Mount temporary directories in
emptydir volumes
[ https://issues.apache.org/jira/browse/SPARK-24137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yinan Li updated SPARK-24137:
-----------------------------
Fix Version/s: 2.3.1
> [K8s] Mount temporary directories in emptydir volumes
> -----------------------------------------------------
>
> Key: SPARK-24137
> URL: https://issues.apache.org/jira/browse/SPARK-24137
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes
> Affects Versions: 2.3.0
> Reporter: Matt Cheah
> Assignee: Matt Cheah
> Priority: Major
> Fix For: 2.3.1, 3.0.0
>
>
> Currently the Spark local directories do not get any volumes and volume mounts, which means we're writing Spark shuffle and cache contents to the file system mounted by Docker. This can be terribly inefficient. We should use emptydir volumes for these directories instead for significant performance improvements.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org