Re: Machine Learning on Flink - Next steps

[Jinkui Shi <sh...@163.com>]

Hi, all

I prefer online learning than offline training.
Offline learning is the basic ability. I think ML in Flink, the more obvious advantage is streaming process.

One suggestion is that FLinkML can define some API to support tensor-flow, mxnet and other ps framework, like apache beam’s mode.

There will be more and more ml framework. We can’t support all of them, but can define the common API to adapter.

Maybe we can start to discuss how to implement such ml framework and its design doc and API design.

Best regards,
Jinkui Shi

> 在 2017年3月31日，下午7:41，Adarsh <ad...@thirdwatch.ai> 写道：
> 
> Thank you,. 
> 
> I vote for: 
> 1) Offline learning with the batch API  
> 2) Low-latency prediction serving -> Online learning 
> 
> In details: 
> 1) Without ML Flink can never become the de-facto streaming engine. 
> 
> 2) Flink is a part of production ecosystem, and  production systems require
> ML support.
> 
> a. Offline training should be supported, because typically most of ML
> algorithms 
> are for batch training. 
> b. Model lifecycle should be supported: 
> ETL+transformation+training+scoring+exploitation quality monitoring 
> 
> I understand that batch world is full of competitors, however training in
> batch and fast execution online
> can be very useful and can give Flink a edge, online learning is also
> desirable however with a lower priority.
> 
> We migrated from Spark to Flink and we love Flink however in absence of good
> ML suppoer we may have to move back to Spark.
> 
> 
> 
> --
> View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Machine-Learning-on-Flink-Next-steps-tp16334p16874.html
> Sent from the Apache Flink Mailing List archive. mailing list archive at Nabble.com.