Calling ALS-MlLib from desktop application/ Training ALS

[Saurabh Agrawal <sa...@markit.com>]

Requesting guidance on my queries in trail email.



-----Original Message-----
From: Saurabh Agrawal
Sent: Saturday, December 13, 2014 07:06 PM GMT Standard Time
To: user@spark.apache.org
Subject: Building Desktop application for ALS-MlLib/ Training ALS




Hi,



I am a new bee in spark and scala world



I have been trying to implement Collaborative filtering using MlLib supplied out of the box with Spark and Scala



I have 2 problems



1.       The best model was trained with rank = 20 and lambda = 5.0, and numIter = 10, and its RMSE on the test set is 25.718710831912485. The best model improves the baseline by 18.29%. Is there a scientific way in which RMSE could be brought down? What is a descent acceptable value for RMSE?

2.       I picked up the Collaborative filtering algorithm from http://ampcamp.berkeley.edu/5/exercises/movie-recommendation-with-mllib.html and executed the given code with my dataset. Now, I want to build a desktop application around it.

a.       What is the best language to do this Java/ Scala? Any possibility to do this using C#?

b.      Can somebody please share any relevant documents/ source or any helper links to help me get started on this?



Your help is greatly appreciated



Thanks!!



Regards,

Saurabh Agrawal

________________________________
This e-mail, including accompanying communications and attachments, is strictly confidential and only for the intended recipient. Any retention, use or disclosure not expressly authorised by Markit is prohibited. This email is subject to all waivers and other terms at the following link: http://www.markit.com/en/about/legal/email-disclaimer.page

Please visit http://www.markit.com/en/about/contact/contact-us.page? for contact information on our offices worldwide.

MarkitSERV Limited has its registered office located at Level 4, Ropemaker Place, 25 Ropemaker Street, London, EC2Y 9LY and is authorized and regulated by the Financial Conduct Authority with registration number 207294

Re: Calling ALS-MlLib from desktop application/ Training ALS

[Krishna Sankar <ks...@gmail.com>]

a) There is no absolute RSME - it depends on the domain. Also RSME is the
error based on what you have seen so far, a snapshot of a slice of the
domain.
b) My suggestion is put the system in place, see what happens when users
interact with the system and then you can think of reducing the RSME as
needed. For all you know, RSME could go up with another set of data
c) I would prefer Scala, but Java would work as well.
d) For a desktop app, you have two ways to go.
    Either run Spark in local machine and build an app or
    Have Spark run in a server/cluster and build a browser app. This
depends on the data size and scaling requirements.
e) I haven't seen any C# interfaces. Might be a good request candidate.
Cheers
<k/>

On Sat, Dec 13, 2014 at 7:17 PM, Saurabh Agrawal <saurabh.agrawal@markit.com
> wrote:
>
>
> Requesting guidance on my queries in trail email.
>
>
>
> -----Original Message-----
> *From: *Saurabh Agrawal
> *Sent: *Saturday, December 13, 2014 07:06 PM GMT Standard Time
> *To: *user@spark.apache.org
> *Subject: *Building Desktop application for ALS-MlLib/ Training ALS
>
>
>
> Hi,
>
>
>
> I am a new bee in spark and scala world
>
>
>
> I have been trying to implement Collaborative filtering using MlLib
> supplied out of the box with Spark and Scala
>
>
>
> I have 2 problems
>
>
>
> 1.       The best model was trained with rank = 20 and lambda = 5.0, and
> numIter = 10, and its RMSE on the test set is 25.718710831912485. The best
> model improves the baseline by 18.29%. Is there a scientific way in which
> RMSE could be brought down? What is a descent acceptable value for RMSE?
>
> 2.       I picked up the Collaborative filtering algorithm from
> http://ampcamp.berkeley.edu/5/exercises/movie-recommendation-with-mllib.html
> and executed the given code with my dataset. Now, I want to build a
> desktop application around it.
>
> a.       What is the best language to do this Java/ Scala? Any
> possibility to do this using C#?
>
> b.      Can somebody please share any relevant documents/ source or any
> helper links to help me get started on this?
>
>
>
> Your help is greatly appreciated
>
>
>
> Thanks!!
>
>
>
> Regards,
>
> Saurabh Agrawal
>
> ------------------------------
> This e-mail, including accompanying communications and attachments, is
> strictly confidential and only for the intended recipient. Any retention,
> use or disclosure not expressly authorised by Markit is prohibited. This
> email is subject to all waivers and other terms at the following link:
> http://www.markit.com/en/about/legal/email-disclaimer.page
>
> Please visit http://www.markit.com/en/about/contact/contact-us.page? for
> contact information on our offices worldwide.
>
> MarkitSERV Limited has its registered office located at Level 4, Ropemaker
> Place, 25 Ropemaker Street, London, EC2Y 9LY and is authorized and
> regulated by the Financial Conduct Authority with registration number 207294
>