[jira] [Updated] (SPARK-47475) Jars Download from Driver under K8s Cluster Mode Caused Executor Scalability Issue

["Jiale Tan (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-47475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jiale Tan updated SPARK-47475:
------------------------------
    Summary: Jars Download from Driver under K8s Cluster Mode Caused Executor Scalability Issue  (was: Make Executor Jars Download from Driver Optional for K8s Cluster Mode Driver)

> Jars Download from Driver under K8s Cluster Mode Caused Executor Scalability Issue
> ----------------------------------------------------------------------------------
>
>                 Key: SPARK-47475
>                 URL: https://issues.apache.org/jira/browse/SPARK-47475
>             Project: Spark
>          Issue Type: Improvement
>          Components: Deploy, Kubernetes, Spark Core
>    Affects Versions: 3.4.0, 3.5.0
>            Reporter: Jiale Tan
>            Priority: Major
>
> Under K8s cluster deployment mode, all the jars, including primary resource jar, jars from {{--jars}} or {{spark.jars}}, will be downloaded to driver local and then served to executors through file server running on driver.
> When jars are big and the application requests a lot of executors, the massive concurrent jars download from the driver will cause network saturation. In this case, the executors jar download will timeout, causing executors to be terminated. From user point of view, the application is trapped in the loop of massive executor loss and re-provision, but never gets enough live executors as requested, which leads to job SLA breach or sometimes job failure.
> To mitigate issue mentioned above, we should allow executors to download remote jars directly from more scalable storage, like s3 or hdfs, instead of going through driver. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org