MLLib: Saving and loading a model

[Sameer Tilak <ss...@live.com>]

Hi All,Resending this:
I am using LinearRegressionWithSGD and then I save the model weights and intercept. File that contains weights have this format:
1.204550.13560.000456......
Intercept is 0 since I am using train not setting the intercept so it can be ignored for the moment. I would now like to initialize a new model object and using these saved weights from the above file. We are using CDH 5.1
Something along these lines:
val weights = sc.textFile("linear-weights");val model = new LinearRegressionWithSGD(weights);
then use is as:
val valuesAndPreds = testData.map { point =>  val prediction = model.predict(point.features)  (point.label, prediction)}

Any pointers to how do I do that?
 		 	   		  

Re: MLLib: Saving and loading a model

[Jaonary Rabarisoa <ja...@gmail.com>]

Hi,

There's is a ongoing work on model export
https://www.github.com/apache/spark/pull/3062

For now, since LinearRegression is serializable you can save it as object
file :

sc.saveAsObjectFile(Seq(model))

then

val model = sc.objectFile[LinearRegresionWithSGD]("path").first

model.predict(...)




On Mon, Dec 15, 2014 at 11:21 PM, Sameer Tilak <ss...@live.com> wrote:
>
> Hi All,
> Resending this:
>
> I am using LinearRegressionWithSGD and then I save the model weights and
> intercept.
> File that contains weights have this format:
>
> 1.20455
> 0.1356
> 0.000456
> ......
>
> Intercept is 0 since I am using train not setting the intercept so it can
> be ignored for the moment. I would now like to initialize a new model
> object and using these saved weights from the above file. We are using CDH
> 5.1
>
> Something along these lines:
>
> val weights = sc.textFile("linear-weights");
> val model = new LinearRegressionWithSGD(weights);
>
> then use is as:
>
> val valuesAndPreds = testData.map { point =>
>   val prediction = model.predict(point.features)
>   (point.label, prediction)
> }
>
>
> Any pointers to how do I do that?
>
>