You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Christophe Salperwyck <ch...@gmail.com> on 2018/02/21 18:38:26 UTC
Machine Learning: Flink and MOA
Hi guys,
I know there is FlinkML to do some machine learning with Flink but it works
on DataSet and not on DataStream, there is also SAMOA which can run on
Flink but I find it a bit too complicated.
I wanted to see if it would be easy to plug directly MOA on Flink and tried
to present it in the DataKRK meetup, but I didn't have time at the end of
the presentation... Nevertheless I spent a bit of time plugging Flink and
MOA and I thought it might be worth sharing it in case it would be
interesting for someone. I also take this opportunity to get some feedback
on it from people in the Flink community if they have a bit of time to
review it.
Here is the code:
https://github.com/csalperwyck/moa-flink-ozabag-example
https://github.com/csalperwyck/moa-flink-traintest-example
Many Flink methods were very convenient to plug these 2 tools :-)
Keep the good work!
Cheers,
Christophe
PS: if some people are in bigdatatechwarsaw and interested, we can discuss
tomorrow :-)
Re: Machine Learning: Flink and MOA
Posted by Christophe Salperwyck <ch...@gmail.com>.
Hello Theodore,
Glad to hear that there is an interest in plugging MOA with Flink!
Which part/type of classifiers of MOA would you want to plug with Flink?
Let me know if you want to discuss in more details.
I guess some windowing function of MOA would be better implemented as Flink
Windows (performance evaluation I would say at first). We would need to
speak with Albert to see how this could be handled (change some MOA code?).
Regards,
Christophe
2018-02-24 1:03 GMT+01:00 Theodore Vasiloudis <
theodoros.vasiloudis@gmail.com>:
> Hello Christophe,
>
> That's very interesting, I've been working with MOA/SAMOA recently and was
> considering if we could create some
> easy integration with Flink.
>
> I have a Master student this year that could do some work on this,
> hopefully we can create something interesting
> there.
>
> Regards,
> Theodore
>
> On Wed, Feb 21, 2018 at 7:38 PM, Christophe Salperwyck <
> christophe.salperwyck@gmail.com> wrote:
>
>> Hi guys,
>>
>> I know there is FlinkML to do some machine learning with Flink but it
>> works on DataSet and not on DataStream, there is also SAMOA which can run
>> on Flink but I find it a bit too complicated.
>>
>> I wanted to see if it would be easy to plug directly MOA on Flink and
>> tried to present it in the DataKRK meetup, but I didn't have time at the
>> end of the presentation... Nevertheless I spent a bit of time plugging
>> Flink and MOA and I thought it might be worth sharing it in case it would
>> be interesting for someone. I also take this opportunity to get some
>> feedback on it from people in the Flink community if they have a bit of
>> time to review it.
>>
>> Here is the code:
>> https://github.com/csalperwyck/moa-flink-ozabag-example
>> https://github.com/csalperwyck/moa-flink-traintest-example
>>
>> Many Flink methods were very convenient to plug these 2 tools :-)
>>
>> Keep the good work!
>>
>> Cheers,
>> Christophe
>> PS: if some people are in bigdatatechwarsaw and interested, we can
>> discuss tomorrow :-)
>>
>
>
Re: Machine Learning: Flink and MOA
Posted by Theodore Vasiloudis <th...@gmail.com>.
Hello Christophe,
That's very interesting, I've been working with MOA/SAMOA recently and was
considering if we could create some
easy integration with Flink.
I have a Master student this year that could do some work on this,
hopefully we can create something interesting
there.
Regards,
Theodore
On Wed, Feb 21, 2018 at 7:38 PM, Christophe Salperwyck <
christophe.salperwyck@gmail.com> wrote:
> Hi guys,
>
> I know there is FlinkML to do some machine learning with Flink but it
> works on DataSet and not on DataStream, there is also SAMOA which can run
> on Flink but I find it a bit too complicated.
>
> I wanted to see if it would be easy to plug directly MOA on Flink and
> tried to present it in the DataKRK meetup, but I didn't have time at the
> end of the presentation... Nevertheless I spent a bit of time plugging
> Flink and MOA and I thought it might be worth sharing it in case it would
> be interesting for someone. I also take this opportunity to get some
> feedback on it from people in the Flink community if they have a bit of
> time to review it.
>
> Here is the code:
> https://github.com/csalperwyck/moa-flink-ozabag-example
> https://github.com/csalperwyck/moa-flink-traintest-example
>
> Many Flink methods were very convenient to plug these 2 tools :-)
>
> Keep the good work!
>
> Cheers,
> Christophe
> PS: if some people are in bigdatatechwarsaw and interested, we can discuss
> tomorrow :-)
>