[GitHub] [spark] tgravescs opened a new pull request #24731: [SPARK-27725][Core] Add an example discovery Script for GPU resources

[GitBox <gi...@apache.org>]

tgravescs opened a new pull request #24731: [SPARK-27725][Core] Add an example discovery Script for GPU resources
URL: https://github.com/apache/spark/pull/24731
 
 
   ## What changes were proposed in this pull request?
   
   Example GPU resource discovery script that can be used with Nvidia GPUs and passed into SPARK via spark.{driver/executor}.resource.gpu.discoveryScript 
   
   For example:
   ./bin/spark-shell --master yarn --deploy-mode client --driver-memory 1g  --conf spark.yarn.am.memory=3g --num-executors 1 --executor-memory 1g  --conf spark.driver.resource.gpu.count=2  --executor-cores 1 --conf spark.driver.resource.gpu.discoveryScript=/home/tgraves/workspace/tgravescs-spark/examples/src/main/resources/getGpusResources.sh  --conf spark.executor.resource.gpu.count=1 --conf spark.task.resource.gpu.count=1 --conf spark.executor.resource.gpu.discoveryScript=/home/tgraves/workspace/tgravescs-spark/examples/src/main/resources/getGpusResources.sh
   
   ## How was this patch tested?
   
   Manually tested local cluster mode and yarn mode. Tested on a node with 8 GPUs and one with 2 GPUs.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org