about spark and using machine learning model

[Hoai-Thu Vuong <th...@gmail.com>]

Hello everybody!

I'm getting started with spark and mllib. I'm successful in building a
small cluster and follow the tutorial. However, I would like to ask about
how to use the model, which is trained by mllib. I understand that, with
data we can training the model such as Classifier model, then use it to
classify new input. Is there any case study to build a service upon spark
or hdfs and using model (trained by above steps) and give output to user
(class of input data). Thank you very much!



-- 
Thu.

Re: about spark and using machine learning model

[Julien Naour <ju...@gmail.com>]

You can find in the following presentation a simple example of a clustering
model use to classify new incoming tweet :
https://www.youtube.com/watch?v=sPhyePwo7FA

Regards,
Julien


2014-08-05 7:08 GMT+02:00 Xiangrui Meng <me...@gmail.com>:

> Some extra work is needed to close the loop. One related example is
> streaming linear regression added by Jeremy very recently:
>
>
> https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/mllib/StreamingLinearRegression.scala
>
> You can use a model trained offline to serve a DStream and save the
> predictions (also a DStream) to somewhere, e.g., HDFS or stdout.
>
> Best,
> Xiangrui
>
> On Mon, Aug 4, 2014 at 9:28 PM, Hoai-Thu Vuong <th...@gmail.com> wrote:
> > Hello everybody!
> >
> > I'm getting started with spark and mllib. I'm successful in building a
> small
> > cluster and follow the tutorial. However, I would like to ask about how
> to
> > use the model, which is trained by mllib. I understand that, with data we
> > can training the model such as Classifier model, then use it to classify
> new
> > input. Is there any case study to build a service upon spark or hdfs and
> > using model (trained by above steps) and give output to user (class of
> input
> > data). Thank you very much!
> >
> >
> >
> > --
> > Thu.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Re: about spark and using machine learning model

[Xiangrui Meng <me...@gmail.com>]

Some extra work is needed to close the loop. One related example is
streaming linear regression added by Jeremy very recently:

https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/mllib/StreamingLinearRegression.scala

You can use a model trained offline to serve a DStream and save the
predictions (also a DStream) to somewhere, e.g., HDFS or stdout.

Best,
Xiangrui

On Mon, Aug 4, 2014 at 9:28 PM, Hoai-Thu Vuong <th...@gmail.com> wrote:
> Hello everybody!
>
> I'm getting started with spark and mllib. I'm successful in building a small
> cluster and follow the tutorial. However, I would like to ask about how to
> use the model, which is trained by mllib. I understand that, with data we
> can training the model such as Classifier model, then use it to classify new
> input. Is there any case study to build a service upon spark or hdfs and
> using model (trained by above steps) and give output to user (class of input
> data). Thank you very much!
>
>
>
> --
> Thu.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org