[jira] [Created] (SPARK-28022) k8s pod affinity achieve cloud native friendly auto scaling

["Henry Yu (JIRA)" <ji...@apache.org>]

Henry Yu created SPARK-28022:
--------------------------------

             Summary: k8s pod affinity achieve cloud native friendly auto scaling 
                 Key: SPARK-28022
                 URL: https://issues.apache.org/jira/browse/SPARK-28022
             Project: Spark
          Issue Type: New Feature
          Components: Kubernetes
    Affects Versions: 2.4.3
            Reporter: Henry Yu


Hi, in order to achieve friendly auto scaling , I propose to add a pod affinity feature.

Traditionally, when we use spark in fix size yarn cluster, it make sense to spread container evenly to every node.

But come to cloud native resource manage, we want to release node when we don't need any more.

pod affinity feature can help when we try to place all pod of certain job to some node instead of all node.

we can not use pod template to achieve this, for we must add application id to affinity .

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org