You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Piyush Shrivastava <pi...@yahoo.co.in> on 2016/05/11 13:44:02 UTC
Using FlinkML algorithms in Streaming
Hello all,
I want to perform linear regression using FlinkML's MultipleLinearRegression() function on streaming data.
This function takes a DataSet as an input and I cannot create a DataSet inside the MapFunction of a DataStream. How can I use this function on my DataStream? Thanks and Regards,Piyush Shrivastava
http://webograffiti.com
Re: Using FlinkML algorithms in Streaming
Posted by Simone Robutti <si...@radicalbit.io>.
Actually model portability and persistence is a serious limitation to
practical use of FlinkML in streaming. If you know what you're doing, you
can write a blunt serializer for your model, write it in a file and rebuild
the model stream-side with deserialized informations.
I tried it for an SVM model and there were no obstacles. It's ugly but it
works.
2016-05-11 16:18 GMT+02:00 Márton Balassi <ba...@gmail.com>:
> Currently I am not aware of streaming learners support, you would need to
> implement that yourself at this point.
>
> As for streaming predictors for batch learners I have some preview code
> that you might like. [1]
>
> [1]
> https://github.com/streamline-eu/ML-Pipelines/blob/314e3d940f1f1ac7b762ba96067e13d806476f57/flink-libraries/flink-stream-ml/src/main/scala/org/apache/flink/stream/ml/examples/MLRExample.scala
>
>
>
> On Wed, May 11, 2016 at 3:52 PM, Piyush Shrivastava <piyushjoy@yahoo.co.in
> > wrote:
>
>> Hi Márton,
>>
>> I want to train and get the residuals.
>>
>> Thanks and Regards,
>> Piyush Shrivastava <pi...@webograffiti.com>
>> [image: WeboGraffiti]
>> http://webograffiti.com
>>
>>
>> On Wednesday, 11 May 2016 7:19 PM, Márton Balassi <
>> balassi.marton@gmail.com> wrote:
>>
>>
>> Hey Piyush,
>>
>> Would you like to train or predict on the streaming data?
>>
>> Best,
>>
>> Marton
>>
>> On Wed, May 11, 2016 at 3:44 PM, Piyush Shrivastava <
>> piyushjoy@yahoo.co.in> wrote:
>>
>> Hello all,
>>
>> I want to perform linear regression using FlinkML's
>> MultipleLinearRegression() function on streaming data.
>>
>> This function takes a DataSet as an input and I cannot create a DataSet
>> inside the MapFunction of a DataStream. How can I use this function on my
>> DataStream?
>>
>> Thanks and Regards,
>> Piyush Shrivastava <pi...@webograffiti.com>
>> [image: WeboGraffiti]
>> http://webograffiti.com
>>
>>
>>
>>
>>
>
Re: Using FlinkML algorithms in Streaming
Posted by Márton Balassi <ba...@gmail.com>.
Currently I am not aware of streaming learners support, you would need to
implement that yourself at this point.
As for streaming predictors for batch learners I have some preview code
that you might like. [1]
[1]
https://github.com/streamline-eu/ML-Pipelines/blob/314e3d940f1f1ac7b762ba96067e13d806476f57/flink-libraries/flink-stream-ml/src/main/scala/org/apache/flink/stream/ml/examples/MLRExample.scala
On Wed, May 11, 2016 at 3:52 PM, Piyush Shrivastava <pi...@yahoo.co.in>
wrote:
> Hi Márton,
>
> I want to train and get the residuals.
>
> Thanks and Regards,
> Piyush Shrivastava <pi...@webograffiti.com>
> [image: WeboGraffiti]
> http://webograffiti.com
>
>
> On Wednesday, 11 May 2016 7:19 PM, Márton Balassi <
> balassi.marton@gmail.com> wrote:
>
>
> Hey Piyush,
>
> Would you like to train or predict on the streaming data?
>
> Best,
>
> Marton
>
> On Wed, May 11, 2016 at 3:44 PM, Piyush Shrivastava <piyushjoy@yahoo.co.in
> > wrote:
>
> Hello all,
>
> I want to perform linear regression using FlinkML's
> MultipleLinearRegression() function on streaming data.
>
> This function takes a DataSet as an input and I cannot create a DataSet
> inside the MapFunction of a DataStream. How can I use this function on my
> DataStream?
>
> Thanks and Regards,
> Piyush Shrivastava <pi...@webograffiti.com>
> [image: WeboGraffiti]
> http://webograffiti.com
>
>
>
>
>
Re: Using FlinkML algorithms in Streaming
Posted by Piyush Shrivastava <pi...@yahoo.co.in>.
Hi Márton,
I want to train and get the residuals.
Thanks and Regards,Piyush Shrivastava
http://webograffiti.com
On Wednesday, 11 May 2016 7:19 PM, Márton Balassi <ba...@gmail.com> wrote:
Hey Piyush,
Would you like to train or predict on the streaming data?
Best,
Marton
On Wed, May 11, 2016 at 3:44 PM, Piyush Shrivastava <pi...@yahoo.co.in> wrote:
Hello all,
I want to perform linear regression using FlinkML's MultipleLinearRegression() function on streaming data.
This function takes a DataSet as an input and I cannot create a DataSet inside the MapFunction of a DataStream. How can I use this function on my DataStream? Thanks and Regards,Piyush Shrivastava
http://webograffiti.com
Re: Using FlinkML algorithms in Streaming
Posted by Márton Balassi <ba...@gmail.com>.
Hey Piyush,
Would you like to train or predict on the streaming data?
Best,
Marton
On Wed, May 11, 2016 at 3:44 PM, Piyush Shrivastava <pi...@yahoo.co.in>
wrote:
> Hello all,
>
> I want to perform linear regression using FlinkML's
> MultipleLinearRegression() function on streaming data.
>
> This function takes a DataSet as an input and I cannot create a DataSet
> inside the MapFunction of a DataStream. How can I use this function on my
> DataStream?
>
> Thanks and Regards,
> Piyush Shrivastava <pi...@webograffiti.com>
> [image: WeboGraffiti]
> http://webograffiti.com
>