[jira] [Created] (SPARK-44526) Porting k8s PVC reuse logic to spark standalone

["Faiz Halde (Jira)" <ji...@apache.org>]

Faiz Halde created SPARK-44526:
----------------------------------

             Summary: Porting k8s PVC reuse logic to spark standalone
                 Key: SPARK-44526
                 URL: https://issues.apache.org/jira/browse/SPARK-44526
             Project: Spark
          Issue Type: New Feature
          Components: Shuffle, Spark Core
    Affects Versions: 3.4.1
            Reporter: Faiz Halde


Hi,

This ticket is meant to understand the work that would be involved in porting the PVC reuse feature onto the spark standalone cluster manager

 

The logic in
KubernetesLocalDiskShuffleDataIO
itself is not that much. However when I tried this on the `LocalDiskShuffleExecutorComponents` it was not a successful experiment which suggests there is more to it

I'd like to understand what will be the work involved for this. We'll be more than happy to contribute



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org