You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Carol McDonald <cm...@maprtech.com> on 2015/07/15 22:55:00 UTC
ALS run method versus ALS train versus ALS fit and transform
In the Spark mllib examples MovieLensALS.scala ALS run is used, however in
the movie recommendation with mllib tutorial ALS train is used , What is
the difference, when should you use one versus the other
val model = new ALS()
.setRank(params.rank)
.setIterations(params.numIterations)
.setLambda(params.lambda)
.setImplicitPrefs(params.implicitPrefs)
.setUserBlocks(params.numUserBlocks)
.setProductBlocks(params.numProductBlocks)
.run(training)
val model = ALS.train(training, rank, numIter, lambda)
Also in org.apache.spark.examples.ml , fit and transform is used. Which
one do you recommend using ?
val als = new ALS()
.setUserCol("userId")
.setItemCol("movieId")
.setRank(params.rank)
.setMaxIter(params.maxIter)
.setRegParam(params.regParam)
.setNumBlocks(params.numBlocks)
val model = als.fit(training.toDF())
val predictions = model.transform(test.toDF()).cache()
Re: ALS run method versus ALS train versus ALS fit and transform
Posted by Sean Owen <so...@cloudera.com>.
Yes, just have a look at the method in the source code. It calls "new
ALS()....run()". It's a convenience wrapper only.
On Fri, Jul 17, 2015 at 4:59 PM, Carol McDonald <cm...@maprtech.com> wrote:
>>the "new ALS()...run()" form is underneath both of the first two.
>
> I am not sure what you mean by underneath, so basically the mllib
> ALS()...run() does the same thing as the mllib ALS train() ?
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: ALS run method versus ALS train versus ALS fit and transform
Posted by Carol McDonald <cm...@maprtech.com>.
>the "new ALS()...run()" form is underneath both of the first two.
I am not sure what you mean by underneath, so basically the mllib ALS()...run()
does the same thing as the mllib ALS train() ?
On Wed, Jul 15, 2015 at 2:02 PM, Sean Owen <so...@cloudera.com> wrote:
> The first two examples are from the .mllib API. Really, the "new
> ALS()...run()" form is underneath both of the first two. In the second
> case, you're calling a convenience method that calls something similar
> to the first example.
>
> The second example is from the new .ml "pipelines" API. Similar ideas,
> but a different API.
>
> On Wed, Jul 15, 2015 at 9:55 PM, Carol McDonald <cm...@maprtech.com>
> wrote:
> > In the Spark mllib examples MovieLensALS.scala ALS run is used, however
> in
> > the movie recommendation with mllib tutorial ALS train is used , What is
> the
> > difference, when should you use one versus the other
> >
> > val model = new ALS()
> > .setRank(params.rank)
> > .setIterations(params.numIterations)
> > .setLambda(params.lambda)
> > .setImplicitPrefs(params.implicitPrefs)
> > .setUserBlocks(params.numUserBlocks)
> > .setProductBlocks(params.numProductBlocks)
> > .run(training)
> >
> >
> > val model = ALS.train(training, rank, numIter, lambda)
> >
> > Also in org.apache.spark.examples.ml , fit and transform is used.
> Which one
> > do you recommend using ?
> >
> > val als = new ALS()
> > .setUserCol("userId")
> > .setItemCol("movieId")
> > .setRank(params.rank)
> > .setMaxIter(params.maxIter)
> > .setRegParam(params.regParam)
> > .setNumBlocks(params.numBlocks)
> >
> > val model = als.fit(training.toDF())
> >
> > val predictions = model.transform(test.toDF()).cache()
> >
> >
>
Re: ALS run method versus ALS train versus ALS fit and transform
Posted by Sean Owen <so...@cloudera.com>.
The first two examples are from the .mllib API. Really, the "new
ALS()...run()" form is underneath both of the first two. In the second
case, you're calling a convenience method that calls something similar
to the first example.
The second example is from the new .ml "pipelines" API. Similar ideas,
but a different API.
On Wed, Jul 15, 2015 at 9:55 PM, Carol McDonald <cm...@maprtech.com> wrote:
> In the Spark mllib examples MovieLensALS.scala ALS run is used, however in
> the movie recommendation with mllib tutorial ALS train is used , What is the
> difference, when should you use one versus the other
>
> val model = new ALS()
> .setRank(params.rank)
> .setIterations(params.numIterations)
> .setLambda(params.lambda)
> .setImplicitPrefs(params.implicitPrefs)
> .setUserBlocks(params.numUserBlocks)
> .setProductBlocks(params.numProductBlocks)
> .run(training)
>
>
> val model = ALS.train(training, rank, numIter, lambda)
>
> Also in org.apache.spark.examples.ml , fit and transform is used. Which one
> do you recommend using ?
>
> val als = new ALS()
> .setUserCol("userId")
> .setItemCol("movieId")
> .setRank(params.rank)
> .setMaxIter(params.maxIter)
> .setRegParam(params.regParam)
> .setNumBlocks(params.numBlocks)
>
> val model = als.fit(training.toDF())
>
> val predictions = model.transform(test.toDF()).cache()
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org