You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2022/09/15 04:20:38 UTC

[GitHub] [hadoop] PrabhuJoseph commented on pull request #4884: YARN-11255. Support loading alternative docker client config from system environment

PrabhuJoseph commented on PR #4884:
URL: https://github.com/apache/hadoop/pull/4884#issuecomment-1247563160

   Thanks @ashutoshcipher for the patch. Overall looks good to me except a minor one. 
   
   Could you also document the steps to run an application like Spark using this new environment variable YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG in https://hadoop.apache.org/docs/r3.0.2/hadoop-yarn/hadoop-yarn-site/DockerContainers.html
   
   For Example:
   
   ```
   DOCKER_IMAGE_NAME=hadoop-docker 
   DOCKER_CLIENT_CONFIG=hdfs:///user/hadoop/config.json
   spark-submit --master yarn \
   --deploy-mode cluster \
   --conf spark.executorEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
   --conf spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME \
   --conf spark.executorEnv.YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=$DOCKER_CLIENT_CONFIG \
   --conf spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_TYPE=docker \
   --conf spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=$DOCKER_IMAGE_NAME \
   --conf spark.yarn.appMasterEnv.YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=$DOCKER_CLIENT_CONFIG \
   sparkR.R
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org