You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@dolphinscheduler.apache.org by Junfan Zhang <ju...@outlook.com> on 2022/03/14 05:59:44 UTC

[Feature-Request] Introduce the task plugin of TonY into dolphinscheduler

Hi the community

Do you mind introducing the task plugin of TonY, which is a framework to natively run
deep learning jobs on Apache Hadoop. It currently supports TensorFlow, PyTorch,
MXNet and Horovod.

TonY enables running either single node or distributed training as a Hadoop application.
This native connector, together with other TonY features, aims to run machine learning
 jobs reliably and flexibly.I think if can be supported into dolphinscheduler, it will make
people reuse the existing on-perm Hadoop cluster resources and dolphinscheduler will
also be a new workflow engine that covers the machine learning lifecycle jobs.

Feel free to discuss if having any ideas.

Best
Junfan Zhang