[GitHub] [spark] tgravescs opened a new pull request #24703: [SPARK-27362][k8s] Resource Scheduling support for k8s

[GitBox <gi...@apache.org>]

tgravescs opened a new pull request #24703: [SPARK-27362][k8s]  Resource Scheduling support for k8s
URL: https://github.com/apache/spark/pull/24703
 
 
   ## What changes were proposed in this pull request?
   
   Add ability to map the spark resource configs spark.{executor/driver}.resource.{resourceName} to kubernetes Container builder so that we request resources (gpu,s/fpgas/etc) from kubernetes.
   Note that the spark configs will overwrite any resource configs users put into a pod template.
   I added a generic vendor config which is only used by kubernetes right now.  I intentionally didn't put it into the kubernetes config namespace just to avoid adding more config prefixes. 
   
   I will add more documentation for this under jira SPARK-27492. I think it will be easier to do all at once to get cohesive story.
   
   ## How was this patch tested?
   
   Unit tests and manually testing on k8s cluster. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org