[GitHub] [submarine] tangzhankun commented on pull request #305: SUBMARINE-499. Refactor job API to experiment API

[GitBox <gi...@apache.org>]

tangzhankun commented on pull request #305:
URL: https://github.com/apache/submarine/pull/305#issuecomment-641958322


   @jiwq This refactor is more like a new feature development in 0.5 consisting "ad-hoc" and "predefined" abstraction in the spec. It's a great job. And it involves a huge change to our existing JobSpec which might not be our 0.4 goal.
   
   After offline discussion with Wanqiang, for 0.4 refactor goal, I'd suggest that we just do name changing in JobSpec with minimum changes referring to the new "experiment" design. Here's my suggestion for spec changes. Please see the comment after the field in it.
   
   ```
   {
     "name": "mnist",
     "namespace": "default",
     “type”: “adhoc”, //New added field. Just a string type, no need to abstract ad-hoc and predefined for now
     “Code”: { // New added field
        “syncMode”: “docker”
        “url” : gcr.io/kubeflow-ci/tf-mnist-with-summaries:1.0"
     }
     "librarySpec": { // Our new design is to split this "librarySpec" to "Environment" + "code"
       "name": "TensorFlow", // Move to taskSpec
        “image": "gcr.io/kubeflow-ci/tf-mnist-with-summaries:1.0”,// Move to "code"
       "cmd": "python /var/tf_mnist/mnist_with_summaries.py --log_dir=/train/log --learning_rate=0.01 --batch_size=150", // move to the parent level
       "envVars": {
         "ENV_1": "ENV1"
       }// delete. Only can set envVars in each ps or worker spec
     },
     "cmd": "python /var/tf_mnist/mnist_with_summaries.py --log_dir=/train/log --learning_rate=0.01 --batch_size=150", // New added field
     "taskSpecs": {
       “type”: "TensorFlow" // Come from "LibrarySpec"
       "Ps": {
         "name": "tensorflow",
         "replicas": 2,
         "resources": "cpu=4,memory=2048M,nvidia.com/gpu=1"
       },
       "Worker": {
         "name": "tensorflow",
         "replicas": 2,
         "resources": "cpu=4,memory=2048M,nvidia.com/gpu=1"
       }
     }
   ```
   Thoughts? @kevin85421 ,@jiwq , @wangdatan ?
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org