You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@submarine.apache.org by "Kevin Su (Jira)" <ji...@apache.org> on 2021/08/28 07:40:00 UTC

[jira] [Resolved] (SUBMARINE-65) Make PyTorch support HDFS

     [ https://issues.apache.org/jira/browse/SUBMARINE-65?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kevin Su resolved SUBMARINE-65.
-------------------------------
    Resolution: Won't Do

> Make PyTorch support HDFS
> -------------------------
>
>                 Key: SUBMARINE-65
>                 URL: https://issues.apache.org/jira/browse/SUBMARINE-65
>             Project: Apache Submarine
>          Issue Type: Sub-task
>          Components: TonY
>            Reporter: Szilard Nemeth
>            Priority: Major
>
> PyTorch does not support HDFS as a datasource like TensorFlow.
> Found this related issue, but nothing meaningful here: https://github.com/pytorch/pytorch/issues/5867
> I think we should make PyTorch support HDFS by contributing to PyTorch's source code.
> Here is a reference to the TF implementation of HDFS connector: https://github.com/tensorflow/tensorflow/tree/17e49b339b2b9a58ed967c69b7acb714dcd9b465/tensorflow/core/platform/hadoop
> Any other ideas to approach this problem are welcome!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@submarine.apache.org
For additional commands, e-mail: dev-help@submarine.apache.org