You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Faiz Halde (Jira)" <ji...@apache.org> on 2023/05/08 09:01:00 UTC
[jira] [Created] (SPARK-43408) Spark caching in the context of a single job
Faiz Halde created SPARK-43408:
----------------------------------
Summary: Spark caching in the context of a single job
Key: SPARK-43408
URL: https://issues.apache.org/jira/browse/SPARK-43408
Project: Spark
Issue Type: Question
Components: Shuffle
Affects Versions: 3.3.1
Reporter: Faiz Halde
Does caching benefit a spark job with only a single action in it? Spark IIRC already optimizes shuffles by persisting them onto the disk
I am unable to find a counter-example where caching would benefit a job with a single action. In every case I can think of, the shuffle checkpoint acts as a good enough caching mechanism in itself
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org