You are viewing a plain text version of this content. The canonical link for it is here.
Posted to submarine-dev@hadoop.apache.org by "Szilard Nemeth (JIRA)" <ji...@apache.org> on 2019/04/29 12:27:00 UTC
[jira] [Comment Edited] (SUBMARINE-52) Generate Service spec +
launch script for single-node PyTorch learning job
[ https://issues.apache.org/jira/browse/SUBMARINE-52?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16829206#comment-16829206 ]
Szilard Nemeth edited comment on SUBMARINE-52 at 4/29/19 12:26 PM:
-------------------------------------------------------------------
Some notes:
1. PyTorch does not support HDFS so with the first iteration, we only support local files as discussed with [~tangzhankun]
2. Similarly to TF, I added a Dockerfile for PyTorch that includes the Cifar10 model.
3. Testing is in progress on a cluster, will get back with the results
4. Documentation will be handled in a separate jira (as the size of the patch is already big)
was (Author: snemeth):
Some notes:
1. PyTorch does not support HDFS so with the first iteration, we only support local files as discussed with [~tangzhankun]
2. Similarly to TF, I added a Dockerfile for PyTorch that includes the Cifar10 model.
3. Testing is in progress on a cluster, will get back with the results
> Generate Service spec + launch script for single-node PyTorch learning job
> --------------------------------------------------------------------------
>
> Key: SUBMARINE-52
> URL: https://issues.apache.org/jira/browse/SUBMARINE-52
> Project: Hadoop Submarine
> Issue Type: Sub-task
> Reporter: Szilard Nemeth
> Assignee: Szilard Nemeth
> Priority: Major
> Attachments: SUBMARINE-52-2.001.patch, SUBMARINE-52.001.patch, SUBMARINE-52.002.patch
>
>
> Similar to what we have for Tensorflow in {{org.apache.hadoop.yarn.submarine.runtimes.yarnservice.YarnServiceJobSubmitter}}, we need a code that generates Service spec file (json) for PyTorch.
> We also need to take care of the separation of CLI/YAML arguments of TF / PyTorch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)