***UNCHECKED*** [jira] [Created] (YARN-8799) [Submarine] A default directory path in HDFS for "checkout_path"?

["Zhankun Tang (JIRA)" <ji...@apache.org>]

Zhankun Tang created YARN-8799:
----------------------------------

             Summary: [Submarine] A default directory path in HDFS for "checkout_path"?
                 Key: YARN-8799
                 URL: https://issues.apache.org/jira/browse/YARN-8799
             Project: Hadoop YARN
          Issue Type: Sub-task
            Reporter: Zhankun Tang
            Assignee: Zhankun Tang
             Fix For: 3.2.0


It might be more simple for user to use "--checkout_path" if we provide a default path in HDFS for checkout_path. It could be under current staging dir or other place that make sense.
{code:java}
yarn jar $HADOOP_BASE_DIR/home/share/hadoop/yarn/hadoop-yarn-submarine-3.2.0-SNAPSHOT.jar job run \
 -verbose \
 -wait_job_finish \
 -keep_staging_dir \
 --env DOCKER_JAVA_HOME=/usr/lib/jvm/java-8-oracle \
 --env DOCKER_HADOOP_HDFS_HOME=/hadoop-3.2.0-SNAPSHOT \
 --name tf-job-001 \
 --docker_image tangzhankun/tensorflow \
 --input_path hdfs://default/user/yarn/cifar-10-data \
 --worker_resources memory=4G,vcores=2 \
 --worker_launch_cmd "cd /cifar10_estimator && python cifar10_main.py --data-dir=%input_path% --job-dir=%checkpoint_path% --num-gpus=0 --train-steps=5"{code}
Above script should works, but the job failed due to invalid path passed to "--job-dir" per my testing. It should be a URI start with "hdfs://".



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org