回复： Re: Any thoughts making Submarine a separate Apache project?

[〖凯塞林洋〗 <97...@qq.com>]

+1.We are very grateful to Liu Xun for his contribution to the submarine. Now our company (BEIKE finding house) uses the submarine architecture as the scheduling framework for the deep learning platform. It has tried to support the regular framework for the deep learning platform, such as TensorFlow, keras, MXnet, etc., and has used TensorFlow and Keras for distributed training, the current use of the situation,  the submarine framework can be used normally. But if it can be separated, the ability to iterate on the AI ​​platform is faster, and support as a stand-alone project.




------------------ 原始邮件 ------------------
发件人: "dashuiguailuyun@gmail.com"<da...@gmail.com>;
发送时间: 2019年7月19日(星，五) 下午4:39
收件人: "Oliver Hu"<ol...@gmail.com>;"Jeff Zhang"<zj...@gmail.com>;
抄送: "sid yu"<ji...@gmail.com>;"Xun Liu"<li...@hadoop.apache.org>;
主题: Re: Re: Any thoughts making Submarine a separate Apache project?



+1.Submarine is already in use at our company（贝壳找房） and is performing. Looking forward to the next step to provide more features



dashuiguailuyun@gmail.com
 
From: Oliver Hu
Date: 2019-07-19 07:50
To: Jeff Zhang
CC: sid yu; Xun Liu; Hadoop Common; yarn-dev; Hdfs-dev; mapreduce-dev; submarine-dev
Subject: Re: Any thoughts making Submarine a separate Apache project?
+1 (non-binding). Make Submarine a separate project would make it easier to
integrate with other components in the ML pipeline and expand cross
platform.
 
On Thu, Jul 18, 2019 at 2:48 AM Jeff Zhang <zj...@gmail.com> wrote:
 
> +1， This is Jeff Zhang from Zeppelin community.
> Thanks Xun for bring this up. Submarine has been integrated into Zeppelin
> several months ago, and I already see some early adoption of that in China.
> AI is fast growing area, I believe moving into a separate project would be
> helpful for Submarine to catch up with the new trend of AI and release more
> new features quickly than before.
>
>
>
> sid yu <ji...@gmail.com> 于2019年7月18日周四 下午2:06写道：
>
> > +1  We are look forward to it. The idea is great.
> >
> > > On Jul 10, 2019, at 3:34 PM, Xun Liu <li...@apache.org> wrote:
> > >
> > > Hi all,
> > >
> > > This is Xun Liu contributing to the Submarine project for deep learning
> > > workloads running with big data workloads together on Hadoop clusters.
> > >
> > > There are a bunch of integrations of Submarine to other projects are
> > > finished or going on, such as Apache Zeppelin, TonY, Azkaban. The next
> > step
> > > of Submarine is going to integrate with more projects like Apache
> Arrow,
> > > Redis, MLflow, etc. & be able to handle end-to-end machine learning use
> > > cases like model serving, notebook management, advanced training
> > > optimizations (like auto parameter tuning, memory cache optimizations
> for
> > > large datasets for training, etc.), and make it run on other platforms
> > like
> > > Kubernetes or natively on Cloud. LinkedIn also wants to donate TonY
> > project
> > > to Apache so we can put Submarine and TonY together to the same
> codebase
> > > (Page #30.
> > >
> >
> https://www.slideshare.net/xkrogen/hadoop-meetup-jan-2019-tony-tensorflow-on-yarn-and-beyond#30
> > > ).
> > >
> > > This expands the scope of the original Submarine project in exciting
> new
> > > ways. Toward that end, would it make sense to create a separate
> Submarine
> > > project at Apache? This can make faster adoption of Submarine, and
> allow
> > > Submarine to grow to a full-blown machine learning platform.
> > >
> > > There will be lots of technical details to work out, but any initial
> > > thoughts on this?
> > >
> > > Best Regards,
> > > Xun Liu
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >
> >
>
> --
> Best Regards
>
> Jeff Zhang
>