You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by debasish83 <gi...@git.apache.org> on 2014/11/05 02:38:17 UTC

[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

GitHub user debasish83 opened a pull request:

    https://github.com/apache/spark/pull/3098

    [MLLIB] SPARK-4231: Add RankingMetrics to examples.MovieLensALS

    @mengxr @srowen
    
    To validate ALS enhancements as proposed in https://issues.apache.org/jira/browse/SPARK-2426, RMSE along with the RankingMetrics measures are important to look at.  
    
    This PR adds a flag --validateProducts to examples.MovieLensALS.
    
    Default validateProducts is false and we compute RMSE and MAP for test set related to product recommendation.
    
    ./bin/spark-submit --master spark://tusca09lmlvt00c.uswin.ad.vzwcorp.com:7077 --jars /Users/v606014/.m2/repository/com/github/scopt/scopt_2.10/3.2.0/scopt_2.10-3.2.0.jar --total-executor-cores 4 --executor-memory 4g --driver-memory 1g --class org.apache.spark.examples.mllib.MovieLensALS ./examples/target/spark-examples_2.10-1.2.0-SNAPSHOT.jar --kryo --lambda 0.065 hdfs://localhost:8020/sandbox/movielens/
    2014-11-04 17:15:24.262 java[4568:1903] Unable to load realm mapping info from SCDynamicStore
    14/11/04 17:15:24 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    Got 1000209 ratings from 6040 users on 3706 movies.
    Training: 799926, test: 200283.
    Test RMSE = 0.8965005871008247 MAP = 7.438473265235346.
    
    --validateProducts will validate user recommendation for each product
    
    ./bin/spark-submit --master spark://tusca09lmlvt00c.uswin.ad.vzwcorp.com:7077 --jars /Users/v606014/.m2/repository/com/github/scopt/scopt_2.10/3.2.0/scopt_2.10-3.2.0.jar --total-executor-cores 4 --executor-memory 4g --driver-memory 1g --class org.apache.spark.examples.mllib.MovieLensALS ./examples/target/spark-examples_2.10-1.2.0-SNAPSHOT.jar --kryo --lambda 0.065 --validateProducts hdfs://localhost:8020/sandbox/movielens/
    2014-11-04 17:16:18.652 java[4635:1903] Unable to load realm mapping info from SCDynamicStore
    14/11/04 17:16:18 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    Got 1000209 ratings from 6040 users on 3706 movies.
    Training: 800014, test: 200195.
    Test RMSE = 0.8986539583457682 MAP = 12.243775391575324.
    
    Sean,
    
    Are we looking at the right numbers here ? MAP for Movielens dataset is around 12.243. You did similar experiments for oryx/myrrix before...
    
    We can perhaps make the test set generation more intelligent but I went with random sampling for now since I was looking at MAP measure..
    
    For prec@k I am not sure what's the right k number to choose at...I do a sweep over k to choose sweet spot internally.


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/debasish83/spark irmetrics

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/3098.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3098
    
----
commit 9b3951f558e5673eb475c575f14876421b5a3abc
Author: Debasish Das <de...@one.verizon.com>
Date:   2014-11-05T01:23:09Z

    validate user/product on MovieLens dataset through user input and compute map measure along with rmse

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-61896567
  
      [Test build #22953 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22953/consoleFull) for   PR 3098 at commit [`cd3ab31`](https://github.com/apache/spark/commit/cd3ab31cb9b244bae2b45396a6269ed1dc59151b).
     * This patch **fails Scala style tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-88291470
  
    @mengxr I also added 2 test-cases for batch predict APIs. These features are useful if users are interested in computing MAP measures...Let me know if I move the function computeRankingMetrics and computeRMSE to the companion class of ml.recommendation.ALS ? Currently both of them are in examples...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-61750597
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22910/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-61750329
  
      [Test build #22910 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22910/consoleFull) for   PR 3098 at commit [`9b3951f`](https://github.com/apache/spark/commit/9b3951f558e5673eb475c575f14876421b5a3abc).
     * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27707370
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -138,14 +141,122 @@ class MatrixFactorizationModel(
       }
     
       private def recommend(
    -      recommendToFeatures: Array[Double],
    -      recommendableFeatures: RDD[(Int, Array[Double])],
    -      num: Int): Array[(Int, Double)] = {
    -    val scored = recommendableFeatures.map { case (id,features) =>
    -      (id, blas.ddot(features.length, recommendToFeatures, 1, features, 1))
    +    recommendToFeatures: Array[Double],
    +    recommendableFeatures: RDD[(Int, Array[Double])],
    +    num: Int): Array[(Int, Double)] = {
    +    val recommendToVector = Vectors.dense(recommendToFeatures)
    +    val scored = recommendableFeatures.map {
    +      case (id, features) =>
    +        (id, BLAS.dot(recommendToVector, Vectors.dense(features)))
    --- End diff --
    
    and same here. Let's keep this code untouched, which is irrelevant to this PR.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-89729777
  
    agreed with the implicit MAP calculation....For netflix dataset, I got 0.014...May be I need to use a better regularization...was that 0.05-0.1 number from using lambda = 0.065 ? 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-88311843
  
      [Test build #29517 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29517/consoleFull) for   PR 3098 at commit [`ee99571`](https://github.com/apache/spark/commit/ee9957144bc2d145c91fc4a4b894ccd2ee6bc2b9).
     * This patch **fails Scala style tests**.
     * This patch **does not merge cleanly**.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27707367
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -70,26 +71,28 @@ class MatrixFactorizationModel(
     
       /** Predict the rating of one user for one product. */
       def predict(user: Int, product: Int): Double = {
    -    val userVector = userFeatures.lookup(user).head
    -    val productVector = productFeatures.lookup(product).head
    -    blas.ddot(userVector.length, userVector, 1, productVector, 1)
    +    val userVector = Vectors.dense(userFeatures.lookup(user).head)
    +    val productVector = Vectors.dense(productFeatures.lookup(product).head)
    +    BLAS.dot(userVector, productVector)
       }
     
       /**
    -    * Predict the rating of many users for many products.
    -    * The output RDD has an element per each element in the input RDD (including all duplicates)
    -    * unless a user or product is missing in the training set.
    -    *
    -    * @param usersProducts  RDD of (user, product) pairs.
    -    * @return RDD of Ratings.
    -    */
    +   * Predict the rating of many users for many products.
    +   * The output RDD has an element per each element in the input RDD (including all duplicates)
    +   * unless a user or product is missing in the training set.
    +   *
    +   * @param usersProducts  RDD of (user, product) pairs.
    +   * @return RDD of Ratings.
    +   */
       def predict(usersProducts: RDD[(Int, Int)]): RDD[Rating] = {
    -    val users = userFeatures.join(usersProducts).map{
    +    val users = userFeatures.join(usersProducts).map {
           case (user, (uFeatures, product)) => (product, (user, uFeatures))
         }
         users.join(productFeatures).map {
           case (product, ((user, uFeatures), pFeatures)) =>
    -        Rating(user, product, blas.ddot(uFeatures.length, uFeatures, 1, pFeatures, 1))
    +        val userVector = Vectors.dense(uFeatures)
    +        val productVector = Vectors.dense(pFeatures)
    +        Rating(user, product, BLAS.dot(userVector, productVector))
    --- End diff --
    
    Same here,


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-89083630
  
    @debasish83 The code style still need fixes. On the high level, I think having variable `num` is not necessary. Let's do a global num in this PR and see whether users ask for variable `num`. @coderxiang added `topByKey` recently, so it would be nice if we can reuse the implementation in couple places.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27528120
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -171,18 +175,62 @@ object MovieLensALS {
     
         println(s"Test RMSE = $rmse.")
     
    +    if (params.validateRecommendation) {
    +      val (map, users) = computeRankingMetrics(model,
    +        training, test, numMovies.toInt)
    +      println(s"Test users $users MAP $map")
    +    }
    +    
         sc.stop()
       }
     
       /** Compute RMSE (Root Mean Squared Error). */
       def computeRmse(model: MatrixFactorizationModel, data: RDD[Rating], implicitPrefs: Boolean) = {
    -
    -    def mapPredictedRating(r: Double) = if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    -
         val predictions: RDD[Rating] = model.predict(data.map(x => (x.user, x.product)))
    -    val predictionsAndRatings = predictions.map{ x =>
    -      ((x.user, x.product), mapPredictedRating(x.rating))
    +    val predictionsAndRatings = predictions.map { x =>
    +      ((x.user, x.product), mapPredictedRating(x.rating, implicitPrefs))
         }.join(data.map(x => ((x.user, x.product), x.rating))).values
         math.sqrt(predictionsAndRatings.map(x => (x._1 - x._2) * (x._1 - x._2)).mean())
       }
    +
    +  def mapPredictedRating(r: Double, implicitPrefs: Boolean) = {
    +    if (implicitPrefs) math.max(math.min(r, 1.0), 0.0)
    +    else r
    +  }
    +
    +  /**
    +   * Compute MAP (Mean Average Precision) statistics for top N product Recommendation
    +   */
    +  def computeRankingMetrics(model: MatrixFactorizationModel,
    +    train: RDD[Rating], test: RDD[Rating], n: Int) = {
    --- End diff --
    
    added


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by MLnick <gi...@git.apache.org>.
Github user MLnick commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r20002546
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -165,22 +169,60 @@ object MovieLensALS {
           .setProductBlocks(params.numProductBlocks)
           .run(training)
     
    -    val rmse = computeRmse(model, test, params.implicitPrefs)
    -
    -    println(s"Test RMSE = $rmse.")
    -
    +    val (rmse, userMap, productMap) = 
    +      computeRecommendationMetrics(model, test, params.implicitPrefs)
    +    
    +    println(s"Test RMSE = $rmse user MAP = $userMap product MAP = $productMap.")
    +    
         sc.stop()
       }
    -
    -  /** Compute RMSE (Root Mean Squared Error). */
    -  def computeRmse(model: MatrixFactorizationModel, data: RDD[Rating], implicitPrefs: Boolean) = {
    -
    -    def mapPredictedRating(r: Double) = if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    -
    +  
    +  /**  
    +   * Threshold for predictions are at 0.5
    +   */
    +  def mapPredictedRating(r: Double, implicitPrefs: Boolean) = {
    +    if (implicitPrefs) math.max(math.min(r, 1.0), 0.0)
    +    else math.max(round(r), 0.0)
    +  }
    +  
    +  /**  
    +   * Compute MAP (Mean Average Precision) statistics
    +   */
    +  def computeMap(predictedAndLabels: RDD[(Int, (Double, Double))]) = {
    +     val ranking = predictedAndLabels.groupByKey.map {
    +      case (user, entries) => {
    +        val predictionValues = entries.toArray
    --- End diff --
    
    I was going to comment on this point too - MAP has a max of 1.0.
    
    The input to `RankingMetrics` should be RDD[(predicted IDs array), (ground truth IDs array)], where the predictions are ordered by score (position matters for avg precision at K).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r24955624
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -18,14 +18,14 @@
     package org.apache.spark.examples.mllib
     
     import scala.collection.mutable
    -
     import org.apache.log4j.{Level, Logger}
     import scopt.OptionParser
    -
     import org.apache.spark.{SparkConf, SparkContext}
     import org.apache.spark.SparkContext._
     import org.apache.spark.mllib.recommendation.{ALS, MatrixFactorizationModel, Rating}
     import org.apache.spark.rdd.RDD
    +import org.apache.spark.mllib.evaluation.RankingMetrics
    +import org.jblas.DoubleMatrix
    --- End diff --
    
    organize imports (https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide#SparkCodeStyleGuide-Imports).
    
    Do not use JBLAS because we plan to remove it in 1.4. Use breeze/netlib-java instead.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-61747036
  
    Can one of the admins verify this patch?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-88361354
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29521/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r24955664
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -17,14 +17,20 @@
     
     package org.apache.spark.mllib.recommendation
     
    -import java.lang.{Integer => JavaInteger}
    -
    -import org.jblas.DoubleMatrix
    +import java.lang.{ Integer => JavaInteger }
     
     import org.apache.spark.SparkContext._
    -import org.apache.spark.api.java.{JavaPairRDD, JavaRDD}
    +import org.apache.spark.api.java.{ JavaPairRDD, JavaRDD }
     import org.apache.spark.rdd.RDD
     
    +import org.apache.spark.util.collection.Utils
    +import org.apache.spark.util.BoundedPriorityQueue
    +
    +import scala.Ordering
    --- End diff --
    
    organize imports


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27529218
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -17,14 +17,20 @@
     
     package org.apache.spark.mllib.recommendation
     
    -import java.lang.{Integer => JavaInteger}
    -
    -import org.jblas.DoubleMatrix
    +import java.lang.{ Integer => JavaInteger }
     
     import org.apache.spark.SparkContext._
    -import org.apache.spark.api.java.{JavaPairRDD, JavaRDD}
    +import org.apache.spark.api.java.{ JavaPairRDD, JavaRDD }
     import org.apache.spark.rdd.RDD
     
    +import org.apache.spark.util.collection.Utils
    +import org.apache.spark.util.BoundedPriorityQueue
    +
    +import scala.Ordering
    --- End diff --
    
    By organizing imports you mean same package imports will move to one right ?
    
    Old: 
    
    import org.apache.spark.mllib.linalg.Vectors
    import org.apache.spark.mllib.linalg.Vector
    import org.apache.spark.mllib.linalg.BLAS
    
    New:
    
    import org.apache.spark.mllib.linalg.{Vectors, Vector, BLAS}



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27624736
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -103,13 +109,106 @@ class MatrixFactorizationModel private[mllib] (
         recommend(productFeatures.lookup(product).head, userFeatures, num)
           .map(t => Rating(t._1, product, t._2))
     
    +  /**
    +   * Recommends topK users/products.
    +   *
    +   * @param num how many users to return. The number returned may be less than this.
    +   * @return [Array[Rating]] objects, each of which contains a userID, the given productID and a
    +   *  "score" in the rating field. Each represents one recommended user, and they are sorted
    +   *  by score, decreasing. The first returned is the one predicted to be most strongly
    +   *  recommended to the product. The score is an opaque value that indicates how strongly
    +   *  recommended the user is.
    +   */
    +
    +  /**
    +   * Recommend topK products for all users
    +   */
    +  def recommendProductsForUsers(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = userFeatures.map { x => (x._1, num) }
    +    recommendProductsForUsers(topK)
    +  }
    +
    +  /**
    +   * Recommend topK users for all products
    +   */
    +  def recommendUsersForProducts(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = productFeatures.map { x => (x._1, num) }
    +    recommendUsersForProducts(topK)
    +  }
    +
    +  val ord = Ordering.by[Rating, Double](x => x.rating)
    --- End diff --
    
    I mean `ord`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27707377
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -138,14 +141,122 @@ class MatrixFactorizationModel(
       }
     
       private def recommend(
    -      recommendToFeatures: Array[Double],
    -      recommendableFeatures: RDD[(Int, Array[Double])],
    -      num: Int): Array[(Int, Double)] = {
    -    val scored = recommendableFeatures.map { case (id,features) =>
    -      (id, blas.ddot(features.length, recommendToFeatures, 1, features, 1))
    +    recommendToFeatures: Array[Double],
    +    recommendableFeatures: RDD[(Int, Array[Double])],
    +    num: Int): Array[(Int, Double)] = {
    +    val recommendToVector = Vectors.dense(recommendToFeatures)
    +    val scored = recommendableFeatures.map {
    +      case (id, features) =>
    +        (id, BLAS.dot(recommendToVector, Vectors.dense(features)))
         }
         scored.top(num)(Ordering.by(_._2))
       }
    +
    +  /**
    +   * Recommends topK products for all users
    +   *
    +   * @param num how many products to return for every user.
    +   * @return [(Int, Array[Rating])] objects, where every tuple contains a userID and an array of
    +   * rating objects which contains the same userId, recommended productID and a "score" in the
    +   * rating field. Semantics of score is same as recommendProducts API
    +   */
    +  def recommendProductsForUsers(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = userFeatures.map { x => (x._1, num) }
    +    recommendProductsForUsers(topK)
    +  }
    +
    +  /**
    +   * Recommends topK users for all products
    +   *
    +   * @param num how many users to return for every product.
    +   * @return [(Int, Array[Rating])] objects, where every tuple contains a productID and an array
    +   * of rating objects which contains the recommended userId, same productID and a "score" in the
    +   * rating field. Semantics of score is same as recommendUsers API
    +   */
    +  def recommendUsersForProducts(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = productFeatures.map { x => (x._1, num) }
    +    recommendUsersForProducts(topK)
    +  }
    +
    +  val ord = Ordering.by[Rating, Double](x => x.rating)
    +  case class FeatureTopK(feature: Vector, topK: Int)
    +
    +  /**
    +   * Recommend topK products for users in userTopK RDD
    +   *
    +   * @param userTopK how many products to return for every user in userTopK RDD.
    +   * @return [(Int, Array[Rating])] objects, where every tuple contains a userID and an array
    +   * of rating objects which contains the same userId, recommended productID and a "score" in the
    +   * rating field. Semantics of score is same as recommendProducts API
    +   */
    +  def recommendProductsForUsers(
    +    userTopK: RDD[(Int, Int)]): RDD[(Int, Array[Rating])] = {
    +    val userFeaturesTopK = userFeatures.join(userTopK).map {
    +      case (userId, (userFeature, topK)) =>
    +        (userId, FeatureTopK(Vectors.dense(userFeature), topK))
    +    }
    +
    +    // TO DO: Do a mini-batch on productFeatures.collect if the dimension of rows are big
    +    val productVectors = productFeatures.map {
    +      x => (x._1, Vectors.dense(x._2))
    +    }.collect
    --- End diff --
    
    `collect` -> `collect()` (We use `()` if this is an action.)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-63745215
  
      [Test build #23642 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23642/consoleFull) for   PR 3098 at commit [`3f97c49`](https://github.com/apache/spark/commit/3f97c499004aa58dfa1b51b8d2cbd6e5776f5fb1).
     * This patch **passes all tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-63738984
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23632/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-61749722
  
    test this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-61768877
  
    So, I already find it a bit confusing that there are MLlib examples that actually _reimplement_ the machine learning algorithm instead of using the one in MLlib. This extends that "problem", to me, by reimplementing RMSE. I personally would prefer not to do this, myself.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-62249929
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23091/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r26545110
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -18,14 +18,14 @@
     package org.apache.spark.examples.mllib
     
     import scala.collection.mutable
    -
     import org.apache.log4j.{Level, Logger}
     import scopt.OptionParser
    -
     import org.apache.spark.{SparkConf, SparkContext}
     import org.apache.spark.SparkContext._
     import org.apache.spark.mllib.recommendation.{ALS, MatrixFactorizationModel, Rating}
     import org.apache.spark.rdd.RDD
    +import org.apache.spark.mllib.evaluation.RankingMetrics
    +import org.jblas.DoubleMatrix
    --- End diff --
    
    Removed


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27529681
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -103,13 +109,106 @@ class MatrixFactorizationModel private[mllib] (
         recommend(productFeatures.lookup(product).head, userFeatures, num)
           .map(t => Rating(t._1, product, t._2))
     
    +  /**
    +   * Recommends topK users/products.
    +   *
    +   * @param num how many users to return. The number returned may be less than this.
    +   * @return [Array[Rating]] objects, each of which contains a userID, the given productID and a
    +   *  "score" in the rating field. Each represents one recommended user, and they are sorted
    +   *  by score, decreasing. The first returned is the one predicted to be most strongly
    +   *  recommended to the product. The score is an opaque value that indicates how strongly
    +   *  recommended the user is.
    +   */
    +
    +  /**
    +   * Recommend topK products for all users
    +   */
    +  def recommendProductsForUsers(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = userFeatures.map { x => (x._1, num) }
    +    recommendProductsForUsers(topK)
    +  }
    +
    +  /**
    +   * Recommend topK users for all products
    +   */
    +  def recommendUsersForProducts(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = productFeatures.map { x => (x._1, num) }
    +    recommendUsersForProducts(topK)
    +  }
    +
    +  val ord = Ordering.by[Rating, Double](x => x.rating)
    +  case class FeatureTopK(feature: Vector, topK: Int)
    +
    +  /**
    +   * Recommend topK products for users in userTopK RDD
    +   */
    +  def recommendProductsForUsers(
    +    userTopK: RDD[(Int, Int)]): RDD[(Int, Array[Rating])] = {
    +    val userFeaturesTopK = userFeatures.join(userTopK).map {
    +      case (userId, (userFeature, topK)) =>
    +        (userId, FeatureTopK(Vectors.dense(userFeature), topK))
    +    }
    +    val productVectors = productFeatures.map {
    +      x => (x._1, Vectors.dense(x._2))
    +    }.collect
    +
    +    userFeaturesTopK.map {
    +      case (userId, userFeatureTopK) => {
    +        val predictions = productVectors.map {
    +          case (productId, productVector) =>
    +            Rating(userId, productId,
    +              BLAS.dot(userFeatureTopK.feature, productVector))
    --- End diff --
    
    I will bring in lot of level 3 BLAS in the next PR...I am writing the dgemv and dgemm versions for several of these APIs...For now I will add a TODO


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r24955684
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -103,13 +109,106 @@ class MatrixFactorizationModel private[mllib] (
         recommend(productFeatures.lookup(product).head, userFeatures, num)
           .map(t => Rating(t._1, product, t._2))
     
    +  /**
    +   * Recommends topK users/products.
    +   *
    +   * @param num how many users to return. The number returned may be less than this.
    +   * @return [Array[Rating]] objects, each of which contains a userID, the given productID and a
    +   *  "score" in the rating field. Each represents one recommended user, and they are sorted
    +   *  by score, decreasing. The first returned is the one predicted to be most strongly
    +   *  recommended to the product. The score is an opaque value that indicates how strongly
    +   *  recommended the user is.
    +   */
    +
    +  /**
    +   * Recommend topK products for all users
    +   */
    +  def recommendProductsForUsers(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = userFeatures.map { x => (x._1, num) }
    +    recommendProductsForUsers(topK)
    +  }
    +
    +  /**
    +   * Recommend topK users for all products
    +   */
    +  def recommendUsersForProducts(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = productFeatures.map { x => (x._1, num) }
    +    recommendUsersForProducts(topK)
    +  }
    +
    +  val ord = Ordering.by[Rating, Double](x => x.rating)
    +  case class FeatureTopK(feature: Vector, topK: Int)
    +
    +  /**
    +   * Recommend topK products for users in userTopK RDD
    --- End diff --
    
    Need to doc the input type.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-95009893
  
    @debasish83 Do you have time to update the PR?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-61749713
  
    ok to test


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27525485
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -74,6 +75,9 @@ object MovieLensALS {
           opt[Unit]("implicitPrefs")
             .text("use implicit preference")
             .action((_, c) => c.copy(implicitPrefs = true))
    +      opt[Unit]("validateRecommendation")
    --- End diff --
    
    cleaned up --validateRecommendation to --metrics


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-62247938
  
      [Test build #23090 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23090/consoleFull) for   PR 3098 at commit [`10cbb37`](https://github.com/apache/spark/commit/10cbb37a7881867d801ae6630ffc0d09b3feebf9).
     * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-62670769
  
    @mengxr added recommendAll API to MatrixFactorizationModel and right now the catesian based topK finding is also in the code for validation...
    
    Example run:
    
    ./bin/spark-submit --master spark://TUSCA09LMLVT00C.local:7077 --jars /Users/v606014/.m2/repository/com/github/scopt/scopt_2.10/3.2.0/scopt_2.10-3.2.0.jar --total-executor-cores 4 --executor-memory 4g --driver-memory 1g --class org.apache.spark.examples.mllib.MovieLensALS ./examples/target/spark-examples_2.10-1.2.0-SNAPSHOT.jar --kryo --lambda 0.065 --validateRecommendation 1.0 hdfs://localhost:8020/sandbox/movielens/
    
    Got 1000209 ratings from 6040 users on 3706 movies.
    Training: 800670, test: 199539.
    Test RMSE = 0.8485243993052966.
    Using recommendAll API
    k 20 prec@k 0.04393839019542896
    k 40 prec@k 0.039640609473335565
    k 60 prec@k 0.03763387435133046
    k 80 prec@k 0.0337777409738324
    k 100 prec@k 0.0318681682676383
    k 120 prec@k 0.0318289720658054
    k 140 prec@k 0.030209861354280044
    k 160 prec@k 0.028638415038092092
    k 180 prec@k 0.02780078024364211
    k 200 prec@k 0.027149718449817808
    Test userMapAPI = 0.02964195032393224
    
    Using Cartesian
    k 20 prec@k 0.05635144087446176
    k 40 prec@k 0.052252401457436246
    k 60 prec@k 0.04856188583416142
    k 80 prec@k 0.0453461411063266
    k 100 prec@k 0.04296621397813845
    k 120 prec@k 0.040878602186154356
    k 140 prec@k 0.03914612217858326
    k 160 prec@k 0.03766768797615105
    k 180 prec@k 0.03648559125538258
    k 200 prec@k 0.03540990394170256
    Test userMap = 0.038507998497677914
    
    Results with predictAll and cartesian should match but right now they are not same...debugging it further...
    
    From the JIRA reference https://issues.apache.org/jira/browse/SPARK-3066, implemented 2 ideas:
    1) collect one side (either user or product) and broadcast it as a matrix
    3) use Utils.takeOrdered to find top-k
    
    The third idea 2) use level-3 BLAS to compute inner products will be refactored once dense distributed matrix multiplication comes online...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27707356
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -167,23 +169,66 @@ object MovieLensALS {
           .setProductBlocks(params.numProductBlocks)
           .run(training)
     
    -    val rmse = computeRmse(model, test, params.implicitPrefs)
    -
    -    println(s"Test RMSE = $rmse.")
    +    params.metrics match {
    +      case "rmse" =>
    +        val rmse = computeRmse(model, test, params.implicitPrefs)
    +        println(s"Test RMSE = $rmse")
    +      case "map" =>
    +        val (map, users) = computeRankingMetrics(model, training, test, numMovies.toInt)
    +        println(s"Test users $users MAP $map")
    +      case _ => println(s"Metrics not defined, options are rmse/map")
    +    }
     
         sc.stop()
       }
     
       /** Compute RMSE (Root Mean Squared Error). */
    -  def computeRmse(model: MatrixFactorizationModel, data: RDD[Rating], implicitPrefs: Boolean)
    -    : Double = {
    -
    -    def mapPredictedRating(r: Double) = if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    -
    +  def computeRmse(
    +    model: MatrixFactorizationModel,
    +    data: RDD[Rating],
    +    implicitPrefs: Boolean) : Double = {
         val predictions: RDD[Rating] = model.predict(data.map(x => (x.user, x.product)))
    -    val predictionsAndRatings = predictions.map{ x =>
    -      ((x.user, x.product), mapPredictedRating(x.rating))
    +    val predictionsAndRatings = predictions.map { x =>
    +      ((x.user, x.product), mapPredictedRating(x.rating, implicitPrefs))
         }.join(data.map(x => ((x.user, x.product), x.rating))).values
         math.sqrt(predictionsAndRatings.map(x => (x._1 - x._2) * (x._1 - x._2)).mean())
       }
    +
    +  def mapPredictedRating(r: Double, implicitPrefs: Boolean) = {
    +    if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    +  }
    +  
    +  /** Compute MAP (Mean Average Precision) statistics for top N product Recommendation */
    +  def computeRankingMetrics(
    +    model: MatrixFactorizationModel,
    +    train: RDD[Rating],
    +    test: RDD[Rating],
    +    n: Int) : (Double, Long) = {
    +    val ord = Ordering.by[(Int, Double), Double](x => x._2)
    +
    +    val testUserLabels = test.map {
    --- End diff --
    
    Please use the topByKey implementation to compute top items for users: https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/mllib/rdd/MLPairRDDFunctions.scala


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r24955670
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -17,14 +17,20 @@
     
     package org.apache.spark.mllib.recommendation
     
    -import java.lang.{Integer => JavaInteger}
    -
    -import org.jblas.DoubleMatrix
    +import java.lang.{ Integer => JavaInteger }
     
     import org.apache.spark.SparkContext._
    -import org.apache.spark.api.java.{JavaPairRDD, JavaRDD}
    +import org.apache.spark.api.java.{ JavaPairRDD, JavaRDD }
    --- End diff --
    
    Do not add space.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-62249927
  
      [Test build #23091 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23091/consoleFull) for   PR 3098 at commit [`f38a1b5`](https://github.com/apache/spark/commit/f38a1b59e27907f2aa9bd732c5f9147b738d3a0f).
     * This patch **passes all tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-88346961
  
      [Test build #29521 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29521/consoleFull) for   PR 3098 at commit [`3a0c4eb`](https://github.com/apache/spark/commit/3a0c4eb7f81ee0845f4945d395f6652c965f941b).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r24955679
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -103,13 +109,106 @@ class MatrixFactorizationModel private[mllib] (
         recommend(productFeatures.lookup(product).head, userFeatures, num)
           .map(t => Rating(t._1, product, t._2))
     
    +  /**
    +   * Recommends topK users/products.
    +   *
    +   * @param num how many users to return. The number returned may be less than this.
    +   * @return [Array[Rating]] objects, each of which contains a userID, the given productID and a
    +   *  "score" in the rating field. Each represents one recommended user, and they are sorted
    +   *  by score, decreasing. The first returned is the one predicted to be most strongly
    +   *  recommended to the product. The score is an opaque value that indicates how strongly
    +   *  recommended the user is.
    +   */
    +
    +  /**
    +   * Recommend topK products for all users
    +   */
    +  def recommendProductsForUsers(num: Int): RDD[(Int, Array[Rating])] = {
    --- End diff --
    
    Please check whether the return type is Java-friendly. You can generate the API doc and check the Java one.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r24955690
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -103,13 +109,106 @@ class MatrixFactorizationModel private[mllib] (
         recommend(productFeatures.lookup(product).head, userFeatures, num)
           .map(t => Rating(t._1, product, t._2))
     
    +  /**
    +   * Recommends topK users/products.
    +   *
    +   * @param num how many users to return. The number returned may be less than this.
    +   * @return [Array[Rating]] objects, each of which contains a userID, the given productID and a
    +   *  "score" in the rating field. Each represents one recommended user, and they are sorted
    +   *  by score, decreasing. The first returned is the one predicted to be most strongly
    +   *  recommended to the product. The score is an opaque value that indicates how strongly
    +   *  recommended the user is.
    +   */
    +
    +  /**
    +   * Recommend topK products for all users
    +   */
    +  def recommendProductsForUsers(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = userFeatures.map { x => (x._1, num) }
    +    recommendProductsForUsers(topK)
    +  }
    +
    +  /**
    +   * Recommend topK users for all products
    +   */
    +  def recommendUsersForProducts(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = productFeatures.map { x => (x._1, num) }
    +    recommendUsersForProducts(topK)
    +  }
    +
    +  val ord = Ordering.by[Rating, Double](x => x.rating)
    +  case class FeatureTopK(feature: Vector, topK: Int)
    +
    +  /**
    +   * Recommend topK products for users in userTopK RDD
    +   */
    +  def recommendProductsForUsers(
    +    userTopK: RDD[(Int, Int)]): RDD[(Int, Array[Rating])] = {
    +    val userFeaturesTopK = userFeatures.join(userTopK).map {
    +      case (userId, (userFeature, topK)) =>
    +        (userId, FeatureTopK(Vectors.dense(userFeature), topK))
    +    }
    +    val productVectors = productFeatures.map {
    +      x => (x._1, Vectors.dense(x._2))
    +    }.collect
    +
    +    userFeaturesTopK.map {
    +      case (userId, userFeatureTopK) => {
    +        val predictions = productVectors.map {
    +          case (productId, productVector) =>
    +            Rating(userId, productId,
    +              BLAS.dot(userFeatureTopK.feature, productVector))
    +        }
    +        (userId, Utils.takeOrdered(predictions.iterator,
    +          userFeatureTopK.topK)(ord.reverse).toArray)
    +      }
    +    }
    +  }
    +  
    +  /**
    +   * Recommend topK users for products in productTopK RDD
    +   */
    +  def recommendUsersForProducts(
    +    productTopK: RDD[(Int, Int)]): RDD[(Int, Array[Rating])] = {
    +    val blocks = userFeatures.partitions.size / 2
    +
    +    val productVectors = productFeatures.join(productTopK).map {
    +      case (productId, (productFeature, topK)) =>
    +        (productId, FeatureTopK(Vectors.dense(productFeature), topK))
    +    }.collect()
    +
    +    userFeatures.mapPartitions { items =>
    +      val predictions = productVectors.map {
    +        x => (x._1, new BoundedPriorityQueue[Rating](x._2.topK)(ord.reverse))
    +      }.toMap
    +      while (items.hasNext) {
    +        val (userId, userFeature) = items.next
    +        val userVector = Vectors.dense(userFeature)
    +        for (i <- 0 until productVectors.length) {
    +          val (productId, productFeatureTopK) = productVectors(i)
    +          val predicted = Rating(userId, productId,
    +            BLAS.dot(userVector, productFeatureTopK.feature))
    +          predictions(productId) ++= Iterator.single(predicted)
    +        }
    +      }
    +      predictions.iterator
    +    }.reduceByKey({ (queue1, queue2) =>
    +      queue1 ++= queue2
    +      queue1
    +    }, blocks).map {
    +      case (productId, predictions) =>
    +        (productId, predictions.toArray)
    +    }
    +  }
    +
       private def recommend(
    -      recommendToFeatures: Array[Double],
    -      recommendableFeatures: RDD[(Int, Array[Double])],
    -      num: Int): Array[(Int, Double)] = {
    -    val recommendToVector = new DoubleMatrix(recommendToFeatures)
    -    val scored = recommendableFeatures.map { case (id,features) =>
    -      (id, recommendToVector.dot(new DoubleMatrix(features)))
    +    recommendToFeatures: Array[Double],
    --- End diff --
    
    The previous indentation is correct.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-88311599
  
      [Test build #29517 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29517/consoleFull) for   PR 3098 at commit [`ee99571`](https://github.com/apache/spark/commit/ee9957144bc2d145c91fc4a4b894ccd2ee6bc2b9).
     * This patch **does not merge cleanly**.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-62247955
  
      [Test build #23090 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23090/consoleFull) for   PR 3098 at commit [`10cbb37`](https://github.com/apache/spark/commit/10cbb37a7881867d801ae6630ffc0d09b3feebf9).
     * This patch **fails Scala style tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-61910387
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22957/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r20616949
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -39,14 +39,15 @@ import org.apache.spark.rdd.RDD
     object MovieLensALS {
     
       case class Params(
    -      input: String = null,
    -      kryo: Boolean = false,
    -      numIterations: Int = 20,
    -      lambda: Double = 1.0,
    -      rank: Int = 10,
    -      numUserBlocks: Int = -1,
    -      numProductBlocks: Int = -1,
    -      implicitPrefs: Boolean = false) extends AbstractParams[Params]
    +    input: String = null,
    --- End diff --
    
    4-space indentation is fixed now...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-61901778
  
      [Test build #22957 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22957/consoleFull) for   PR 3098 at commit [`4bbae0f`](https://github.com/apache/spark/commit/4bbae0f248ca8747b47ecf852d5aba19c9b39dab).
     * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27624801
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -39,14 +39,15 @@ import org.apache.spark.rdd.RDD
     object MovieLensALS {
     
       case class Params(
    -      input: String = null,
    -      kryo: Boolean = false,
    -      numIterations: Int = 20,
    -      lambda: Double = 1.0,
    -      rank: Int = 10,
    -      numUserBlocks: Int = -1,
    -      numProductBlocks: Int = -1,
    -      implicitPrefs: Boolean = false) extends AbstractParams[Params]
    +    input: String = null,
    --- End diff --
    
    I think it went back to 2-space indentation. The old code has correct indentation. Please do not change.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27707364
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -167,23 +169,66 @@ object MovieLensALS {
           .setProductBlocks(params.numProductBlocks)
           .run(training)
     
    -    val rmse = computeRmse(model, test, params.implicitPrefs)
    -
    -    println(s"Test RMSE = $rmse.")
    +    params.metrics match {
    +      case "rmse" =>
    +        val rmse = computeRmse(model, test, params.implicitPrefs)
    +        println(s"Test RMSE = $rmse")
    +      case "map" =>
    +        val (map, users) = computeRankingMetrics(model, training, test, numMovies.toInt)
    +        println(s"Test users $users MAP $map")
    +      case _ => println(s"Metrics not defined, options are rmse/map")
    +    }
     
         sc.stop()
       }
     
       /** Compute RMSE (Root Mean Squared Error). */
    -  def computeRmse(model: MatrixFactorizationModel, data: RDD[Rating], implicitPrefs: Boolean)
    -    : Double = {
    -
    -    def mapPredictedRating(r: Double) = if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    -
    +  def computeRmse(
    +    model: MatrixFactorizationModel,
    +    data: RDD[Rating],
    +    implicitPrefs: Boolean) : Double = {
         val predictions: RDD[Rating] = model.predict(data.map(x => (x.user, x.product)))
    -    val predictionsAndRatings = predictions.map{ x =>
    -      ((x.user, x.product), mapPredictedRating(x.rating))
    +    val predictionsAndRatings = predictions.map { x =>
    +      ((x.user, x.product), mapPredictedRating(x.rating, implicitPrefs))
         }.join(data.map(x => ((x.user, x.product), x.rating))).values
         math.sqrt(predictionsAndRatings.map(x => (x._1 - x._2) * (x._1 - x._2)).mean())
       }
    +
    +  def mapPredictedRating(r: Double, implicitPrefs: Boolean) = {
    +    if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    +  }
    +  
    +  /** Compute MAP (Mean Average Precision) statistics for top N product Recommendation */
    +  def computeRankingMetrics(
    +    model: MatrixFactorizationModel,
    +    train: RDD[Rating],
    +    test: RDD[Rating],
    +    n: Int) : (Double, Long) = {
    +    val ord = Ordering.by[(Int, Double), Double](x => x._2)
    +
    +    val testUserLabels = test.map {
    +      x => (x.user, (x.product, x.rating))
    +    }.groupByKey().map {
    +      case (userId, products) =>
    +        val sortedProducts = products.toArray.sorted(ord.reverse)
    +        (userId, sortedProducts.map(_._1))
    +    }
    +
    +    val trainUserLabels = train.map {
    +      x => (x.user, x.product)
    +    }.groupByKey().map{case (userId, products) => (userId, products.toArray)}
    +
    +    val rankings = model.recommendProductsForUsers(n).join(trainUserLabels).map {
    +      case (userId, (pred, train)) => {
    +        val predictedProducts = pred.map(_.product)
    +        val trainSet = train.toSet
    +        (userId, predictedProducts.filterNot { x => trainSet.contains(x) })
    +      }
    +    }.join(testUserLabels).map {
    +      case (user, (pred, lab)) => (pred, lab)
    +    }
    +
    +    val metrics = new RankingMetrics(rankings)
    +    (metrics.meanAveragePrecision, testUserLabels.count)
    --- End diff --
    
    If we use `testUserLabels` twice, we should cache it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27769592
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -167,23 +169,66 @@ object MovieLensALS {
           .setProductBlocks(params.numProductBlocks)
           .run(training)
     
    -    val rmse = computeRmse(model, test, params.implicitPrefs)
    -
    -    println(s"Test RMSE = $rmse.")
    +    params.metrics match {
    +      case "rmse" =>
    +        val rmse = computeRmse(model, test, params.implicitPrefs)
    +        println(s"Test RMSE = $rmse")
    +      case "map" =>
    +        val (map, users) = computeRankingMetrics(model, training, test, numMovies.toInt)
    +        println(s"Test users $users MAP $map")
    +      case _ => println(s"Metrics not defined, options are rmse/map")
    +    }
     
         sc.stop()
       }
     
       /** Compute RMSE (Root Mean Squared Error). */
    -  def computeRmse(model: MatrixFactorizationModel, data: RDD[Rating], implicitPrefs: Boolean)
    -    : Double = {
    -
    -    def mapPredictedRating(r: Double) = if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    -
    +  def computeRmse(
    +    model: MatrixFactorizationModel,
    +    data: RDD[Rating],
    +    implicitPrefs: Boolean) : Double = {
         val predictions: RDD[Rating] = model.predict(data.map(x => (x.user, x.product)))
    -    val predictionsAndRatings = predictions.map{ x =>
    -      ((x.user, x.product), mapPredictedRating(x.rating))
    +    val predictionsAndRatings = predictions.map { x =>
    +      ((x.user, x.product), mapPredictedRating(x.rating, implicitPrefs))
         }.join(data.map(x => ((x.user, x.product), x.rating))).values
         math.sqrt(predictionsAndRatings.map(x => (x._1 - x._2) * (x._1 - x._2)).mean())
       }
    +
    +  def mapPredictedRating(r: Double, implicitPrefs: Boolean) = {
    +    if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    +  }
    +  
    +  /** Compute MAP (Mean Average Precision) statistics for top N product Recommendation */
    +  def computeRankingMetrics(
    +    model: MatrixFactorizationModel,
    +    train: RDD[Rating],
    +    test: RDD[Rating],
    +    n: Int) : (Double, Long) = {
    +    val ord = Ordering.by[(Int, Double), Double](x => x._2)
    +
    +    val testUserLabels = test.map {
    --- End diff --
    
    I will update with topByKey....Is there a better place to move this function ? may be inside ALS object for example ? That way I can add a test-case to guard it ? 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-89729865
  
    That might be a fine score. I remember them being MAP being quite low in general and that 0.1 would be "very good". I don't recall the number for Netflix, so don't take that number as a guide. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-61750593
  
      [Test build #22910 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22910/consoleFull) for   PR 3098 at commit [`9b3951f`](https://github.com/apache/spark/commit/9b3951f558e5673eb475c575f14876421b5a3abc).
     * This patch **fails Scala style tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27707366
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -70,26 +71,28 @@ class MatrixFactorizationModel(
     
       /** Predict the rating of one user for one product. */
       def predict(user: Int, product: Int): Double = {
    -    val userVector = userFeatures.lookup(user).head
    -    val productVector = productFeatures.lookup(product).head
    -    blas.ddot(userVector.length, userVector, 1, productVector, 1)
    +    val userVector = Vectors.dense(userFeatures.lookup(user).head)
    +    val productVector = Vectors.dense(productFeatures.lookup(product).head)
    +    BLAS.dot(userVector, productVector)
    --- End diff --
    
    This change is not necessary. It add more wrappers (and temp objects) and it doesn't save much code.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27707373
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -138,14 +141,122 @@ class MatrixFactorizationModel(
       }
     
       private def recommend(
    -      recommendToFeatures: Array[Double],
    -      recommendableFeatures: RDD[(Int, Array[Double])],
    -      num: Int): Array[(Int, Double)] = {
    -    val scored = recommendableFeatures.map { case (id,features) =>
    -      (id, blas.ddot(features.length, recommendToFeatures, 1, features, 1))
    +    recommendToFeatures: Array[Double],
    +    recommendableFeatures: RDD[(Int, Array[Double])],
    +    num: Int): Array[(Int, Double)] = {
    +    val recommendToVector = Vectors.dense(recommendToFeatures)
    +    val scored = recommendableFeatures.map {
    +      case (id, features) =>
    +        (id, BLAS.dot(recommendToVector, Vectors.dense(features)))
         }
         scored.top(num)(Ordering.by(_._2))
       }
    +
    +  /**
    +   * Recommends topK products for all users
    +   *
    +   * @param num how many products to return for every user.
    +   * @return [(Int, Array[Rating])] objects, where every tuple contains a userID and an array of
    +   * rating objects which contains the same userId, recommended productID and a "score" in the
    +   * rating field. Semantics of score is same as recommendProducts API
    +   */
    +  def recommendProductsForUsers(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = userFeatures.map { x => (x._1, num) }
    --- End diff --
    
    Is it common to use different `num`s? I think it should be sufficient to set `num` globally.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-90740329
  
    @debasish83 In RankingMetrics, the label set doesn't have an ordering. It is either hit or miss at each position.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r24955674
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -35,33 +41,33 @@ import org.apache.spark.rdd.RDD
      *                        and the features computed for this product.
      */
     class MatrixFactorizationModel private[mllib] (
    -    val rank: Int,
    -    val userFeatures: RDD[(Int, Array[Double])],
    -    val productFeatures: RDD[(Int, Array[Double])]) extends Serializable {
    +  val rank: Int,
    +  val userFeatures: RDD[(Int, Array[Double])],
    +  val productFeatures: RDD[(Int, Array[Double])]) extends Serializable {
       /** Predict the rating of one user for one product. */
       def predict(user: Int, product: Int): Double = {
    -    val userVector = new DoubleMatrix(userFeatures.lookup(user).head)
    -    val productVector = new DoubleMatrix(productFeatures.lookup(product).head)
    -    userVector.dot(productVector)
    +    val userVector = Vectors.dense(userFeatures.lookup(user).head)
    --- End diff --
    
    Thanks for switching to BLAS.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-62468213
  
    That sounds about like what I recall. I should warn that 'precision' is entirely dependent on a concept of 'relevant' and 'not relevant' recommendations, which depends on your strategy for holding out data -- recent? top rated? random? So it is not a universal metric. You're also never really sure that "non relevant" recommendations are bad. So 14% can be "pretty good". 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-63736377
  
      [Test build #23642 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23642/consoleFull) for   PR 3098 at commit [`3f97c49`](https://github.com/apache/spark/commit/3f97c499004aa58dfa1b51b8d2cbd6e5776f5fb1).
     * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-98002797
  
    @debasish83 Do you mind me sending you an update?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-88312094
  
      [Test build #29518 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29518/consoleFull) for   PR 3098 at commit [`98fa424`](https://github.com/apache/spark/commit/98fa4243dc6041290bdde51e1e899a8be7576470).
     * This patch **fails Scala style tests**.
     * This patch **does not merge cleanly**.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-62247956
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23090/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-62064318
  
    @coderxiang I read the reference paper and I understood the issue...
    
    I thought it as regression metric before but it is not...the predicted value does not matter...the rank of the movieId from predicted set matters...I am updating the PR with following steps (this is focused on user recommendation) if the --validateRecommedation is set...
    
    1. For every user generate train and test set using (0.8, 0.2) and use RDD.sampleByKey
    2. For every user, the predicted set is of size numProducts...I am using MatrixFactorizationModel.recommendProduct(userId, numProducts) API to generate the predicted set
    3. For every user, the labeled set comes from the test set as computed in Step 1
    4. Once I have these two array for every user, I call RankingMetrics to call meanAveragePrecision


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r20050413
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -135,8 +141,11 @@ object MovieLensALS {
     
         println(s"Got $numRatings ratings from $numUsers users on $numMovies movies.")
     
    -    val splits = ratings.randomSplit(Array(0.8, 0.2))
    -    val training = splits(0).cache()
    +    val fractions = (0 until numUsers.toInt).map(x => (x + 1, 0.8)).toMap
    +    
    +    val training = ratings.map { x => (x.user, x) }.sampleByKey(false, fractions).map { x => x._2 }
    --- End diff --
    
    `sampleByKey` with the same `fraction` for each group is the same as `sample(false, fraction)`. If you use `sampleByKeyExact`, we need to collect the keys to driver, which is expensive. I think we can keep the simple `randomSplit` code, and in evaluation, we use a outer join and define a metric for those not appeared in training.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-61749701
  
    add to whitelist


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r24955654
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -171,18 +175,62 @@ object MovieLensALS {
     
         println(s"Test RMSE = $rmse.")
     
    +    if (params.validateRecommendation) {
    +      val (map, users) = computeRankingMetrics(model,
    +        training, test, numMovies.toInt)
    +      println(s"Test users $users MAP $map")
    +    }
    +    
         sc.stop()
       }
     
       /** Compute RMSE (Root Mean Squared Error). */
       def computeRmse(model: MatrixFactorizationModel, data: RDD[Rating], implicitPrefs: Boolean) = {
    -
    -    def mapPredictedRating(r: Double) = if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    -
         val predictions: RDD[Rating] = model.predict(data.map(x => (x.user, x.product)))
    -    val predictionsAndRatings = predictions.map{ x =>
    -      ((x.user, x.product), mapPredictedRating(x.rating))
    +    val predictionsAndRatings = predictions.map { x =>
    +      ((x.user, x.product), mapPredictedRating(x.rating, implicitPrefs))
         }.join(data.map(x => ((x.user, x.product), x.rating))).values
         math.sqrt(predictionsAndRatings.map(x => (x._1 - x._2) * (x._1 - x._2)).mean())
       }
    +
    +  def mapPredictedRating(r: Double, implicitPrefs: Boolean) = {
    +    if (implicitPrefs) math.max(math.min(r, 1.0), 0.0)
    +    else r
    +  }
    +
    +  /**
    +   * Compute MAP (Mean Average Precision) statistics for top N product Recommendation
    +   */
    +  def computeRankingMetrics(model: MatrixFactorizationModel,
    +    train: RDD[Rating], test: RDD[Rating], n: Int) = {
    +
    +    val ord = Ordering.by[(Int, Double), Double](x => x._2)
    +
    +    val testUserLabels = test.map {
    +      x => (x.user, (x.product, x.rating))
    +    }.groupByKey.map {
    --- End diff --
    
    `.groupByKey()`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-88292172
  
    If we move computeRankingMetrics and computeRMSE to a better place, I can guard it through tests...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r20050274
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -18,14 +18,14 @@
     package org.apache.spark.examples.mllib
     
     import scala.collection.mutable
    -
    -import org.apache.log4j.{Level, Logger}
    +import org.apache.log4j.{ Level, Logger }
    --- End diff --
    
    @debasish83 Please update your IDE to fit Spark Code Style Guide:
    
    https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide
    
    `{ Level, Logger }` -> `{Level, Logger}`
    
    If you are not sure about the code style, please follow the existing code style in the Spark code base.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-89729632
  
    I don't remember from the last time I tried the Netflix dataset but in general MAP is low, like 0.05-0.1, even when recommendations are "good" according to RMSE. 
    
    MAP only requires a ranking and so there is no difference in how you define it when the input is ratings or clicks (explicit vs implicit). he input data set does not need a ranking, just a notion of "relevant or not". The recommendation is relevant / correct if the user actually interacted with it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27707363
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -167,23 +169,66 @@ object MovieLensALS {
           .setProductBlocks(params.numProductBlocks)
           .run(training)
     
    -    val rmse = computeRmse(model, test, params.implicitPrefs)
    -
    -    println(s"Test RMSE = $rmse.")
    +    params.metrics match {
    +      case "rmse" =>
    +        val rmse = computeRmse(model, test, params.implicitPrefs)
    +        println(s"Test RMSE = $rmse")
    +      case "map" =>
    +        val (map, users) = computeRankingMetrics(model, training, test, numMovies.toInt)
    +        println(s"Test users $users MAP $map")
    +      case _ => println(s"Metrics not defined, options are rmse/map")
    +    }
     
         sc.stop()
       }
     
       /** Compute RMSE (Root Mean Squared Error). */
    -  def computeRmse(model: MatrixFactorizationModel, data: RDD[Rating], implicitPrefs: Boolean)
    -    : Double = {
    -
    -    def mapPredictedRating(r: Double) = if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    -
    +  def computeRmse(
    +    model: MatrixFactorizationModel,
    +    data: RDD[Rating],
    +    implicitPrefs: Boolean) : Double = {
         val predictions: RDD[Rating] = model.predict(data.map(x => (x.user, x.product)))
    -    val predictionsAndRatings = predictions.map{ x =>
    -      ((x.user, x.product), mapPredictedRating(x.rating))
    +    val predictionsAndRatings = predictions.map { x =>
    +      ((x.user, x.product), mapPredictedRating(x.rating, implicitPrefs))
         }.join(data.map(x => ((x.user, x.product), x.rating))).values
         math.sqrt(predictionsAndRatings.map(x => (x._1 - x._2) * (x._1 - x._2)).mean())
       }
    +
    +  def mapPredictedRating(r: Double, implicitPrefs: Boolean) = {
    +    if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    +  }
    +  
    +  /** Compute MAP (Mean Average Precision) statistics for top N product Recommendation */
    +  def computeRankingMetrics(
    +    model: MatrixFactorizationModel,
    +    train: RDD[Rating],
    +    test: RDD[Rating],
    +    n: Int) : (Double, Long) = {
    +    val ord = Ordering.by[(Int, Double), Double](x => x._2)
    +
    +    val testUserLabels = test.map {
    +      x => (x.user, (x.product, x.rating))
    +    }.groupByKey().map {
    +      case (userId, products) =>
    +        val sortedProducts = products.toArray.sorted(ord.reverse)
    +        (userId, sortedProducts.map(_._1))
    +    }
    +
    +    val trainUserLabels = train.map {
    +      x => (x.user, x.product)
    +    }.groupByKey().map{case (userId, products) => (userId, products.toArray)}
    +
    +    val rankings = model.recommendProductsForUsers(n).join(trainUserLabels).map {
    +      case (userId, (pred, train)) => {
    +        val predictedProducts = pred.map(_.product)
    +        val trainSet = train.toSet
    +        (userId, predictedProducts.filterNot { x => trainSet.contains(x) })
    +      }
    +    }.join(testUserLabels).map {
    --- End diff --
    
    safer to use `rightOuterJoin`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27707362
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -167,23 +169,66 @@ object MovieLensALS {
           .setProductBlocks(params.numProductBlocks)
           .run(training)
     
    -    val rmse = computeRmse(model, test, params.implicitPrefs)
    -
    -    println(s"Test RMSE = $rmse.")
    +    params.metrics match {
    +      case "rmse" =>
    +        val rmse = computeRmse(model, test, params.implicitPrefs)
    +        println(s"Test RMSE = $rmse")
    +      case "map" =>
    +        val (map, users) = computeRankingMetrics(model, training, test, numMovies.toInt)
    +        println(s"Test users $users MAP $map")
    +      case _ => println(s"Metrics not defined, options are rmse/map")
    +    }
     
         sc.stop()
       }
     
       /** Compute RMSE (Root Mean Squared Error). */
    -  def computeRmse(model: MatrixFactorizationModel, data: RDD[Rating], implicitPrefs: Boolean)
    -    : Double = {
    -
    -    def mapPredictedRating(r: Double) = if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    -
    +  def computeRmse(
    +    model: MatrixFactorizationModel,
    +    data: RDD[Rating],
    +    implicitPrefs: Boolean) : Double = {
         val predictions: RDD[Rating] = model.predict(data.map(x => (x.user, x.product)))
    -    val predictionsAndRatings = predictions.map{ x =>
    -      ((x.user, x.product), mapPredictedRating(x.rating))
    +    val predictionsAndRatings = predictions.map { x =>
    +      ((x.user, x.product), mapPredictedRating(x.rating, implicitPrefs))
         }.join(data.map(x => ((x.user, x.product), x.rating))).values
         math.sqrt(predictionsAndRatings.map(x => (x._1 - x._2) * (x._1 - x._2)).mean())
       }
    +
    +  def mapPredictedRating(r: Double, implicitPrefs: Boolean) = {
    +    if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    +  }
    +  
    +  /** Compute MAP (Mean Average Precision) statistics for top N product Recommendation */
    +  def computeRankingMetrics(
    +    model: MatrixFactorizationModel,
    +    train: RDD[Rating],
    +    test: RDD[Rating],
    +    n: Int) : (Double, Long) = {
    +    val ord = Ordering.by[(Int, Double), Double](x => x._2)
    +
    +    val testUserLabels = test.map {
    +      x => (x.user, (x.product, x.rating))
    +    }.groupByKey().map {
    +      case (userId, products) =>
    +        val sortedProducts = products.toArray.sorted(ord.reverse)
    +        (userId, sortedProducts.map(_._1))
    +    }
    +
    +    val trainUserLabels = train.map {
    +      x => (x.user, x.product)
    +    }.groupByKey().map{case (userId, products) => (userId, products.toArray)}
    +
    +    val rankings = model.recommendProductsForUsers(n).join(trainUserLabels).map {
    +      case (userId, (pred, train)) => {
    +        val predictedProducts = pred.map(_.product)
    --- End diff --
    
    `pred.map(_.product).diff(train)`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r24955675
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -103,13 +109,106 @@ class MatrixFactorizationModel private[mllib] (
         recommend(productFeatures.lookup(product).head, userFeatures, num)
           .map(t => Rating(t._1, product, t._2))
     
    +  /**
    +   * Recommends topK users/products.
    --- End diff --
    
    This is JavaDoc style. For normal multiline comments, please use `/*` instead of `/**`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-61749941
  
      [Test build #22909 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22909/consoleFull) for   PR 3098 at commit [`9b3951f`](https://github.com/apache/spark/commit/9b3951f558e5673eb475c575f14876421b5a3abc).
     * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-61749696
  
    cc: @coderxiang


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r24955687
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -103,13 +109,106 @@ class MatrixFactorizationModel private[mllib] (
         recommend(productFeatures.lookup(product).head, userFeatures, num)
           .map(t => Rating(t._1, product, t._2))
     
    +  /**
    +   * Recommends topK users/products.
    +   *
    +   * @param num how many users to return. The number returned may be less than this.
    +   * @return [Array[Rating]] objects, each of which contains a userID, the given productID and a
    +   *  "score" in the rating field. Each represents one recommended user, and they are sorted
    +   *  by score, decreasing. The first returned is the one predicted to be most strongly
    +   *  recommended to the product. The score is an opaque value that indicates how strongly
    +   *  recommended the user is.
    +   */
    +
    +  /**
    +   * Recommend topK products for all users
    +   */
    +  def recommendProductsForUsers(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = userFeatures.map { x => (x._1, num) }
    +    recommendProductsForUsers(topK)
    +  }
    +
    +  /**
    +   * Recommend topK users for all products
    +   */
    +  def recommendUsersForProducts(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = productFeatures.map { x => (x._1, num) }
    +    recommendUsersForProducts(topK)
    +  }
    +
    +  val ord = Ordering.by[Rating, Double](x => x.rating)
    +  case class FeatureTopK(feature: Vector, topK: Int)
    +
    +  /**
    +   * Recommend topK products for users in userTopK RDD
    +   */
    +  def recommendProductsForUsers(
    +    userTopK: RDD[(Int, Int)]): RDD[(Int, Array[Rating])] = {
    +    val userFeaturesTopK = userFeatures.join(userTopK).map {
    +      case (userId, (userFeature, topK)) =>
    +        (userId, FeatureTopK(Vectors.dense(userFeature), topK))
    +    }
    +    val productVectors = productFeatures.map {
    +      x => (x._1, Vectors.dense(x._2))
    +    }.collect
    +
    +    userFeaturesTopK.map {
    +      case (userId, userFeatureTopK) => {
    +        val predictions = productVectors.map {
    +          case (productId, productVector) =>
    +            Rating(userId, productId,
    +              BLAS.dot(userFeatureTopK.feature, productVector))
    --- End diff --
    
    Leave to TODO here for Level 3 BLAS.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27529308
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -35,33 +41,33 @@ import org.apache.spark.rdd.RDD
      *                        and the features computed for this product.
      */
     class MatrixFactorizationModel private[mllib] (
    -    val rank: Int,
    -    val userFeatures: RDD[(Int, Array[Double])],
    -    val productFeatures: RDD[(Int, Array[Double])]) extends Serializable {
    +  val rank: Int,
    --- End diff --
    
    after merge this is fixed


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27525568
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -171,18 +175,62 @@ object MovieLensALS {
     
         println(s"Test RMSE = $rmse.")
     
    +    if (params.validateRecommendation) {
    +      val (map, users) = computeRankingMetrics(model,
    +        training, test, numMovies.toInt)
    +      println(s"Test users $users MAP $map")
    +    }
    +    
         sc.stop()
       }
     
       /** Compute RMSE (Root Mean Squared Error). */
       def computeRmse(model: MatrixFactorizationModel, data: RDD[Rating], implicitPrefs: Boolean) = {
    -
    -    def mapPredictedRating(r: Double) = if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    -
         val predictions: RDD[Rating] = model.predict(data.map(x => (x.user, x.product)))
    -    val predictionsAndRatings = predictions.map{ x =>
    -      ((x.user, x.product), mapPredictedRating(x.rating))
    +    val predictionsAndRatings = predictions.map { x =>
    +      ((x.user, x.product), mapPredictedRating(x.rating, implicitPrefs))
         }.join(data.map(x => ((x.user, x.product), x.rating))).values
         math.sqrt(predictionsAndRatings.map(x => (x._1 - x._2) * (x._1 - x._2)).mean())
       }
    +
    +  def mapPredictedRating(r: Double, implicitPrefs: Boolean) = {
    +    if (implicitPrefs) math.max(math.min(r, 1.0), 0.0)
    --- End diff --
    
    fixed...can be fit in one line


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-64804819
  
    @mengxr can we please review and merge this ? It will greatly help in reporting results from PLSA and other variants of matrix factorization results...should I update the API to use level 3 BLAS? I have used BLAS.dot in this code...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-62465572
  
    @srowen does the MAP numbers for MovieLens dataset looks good to you ? I read on a blog that you got MAP of 14% on MovieLens10M...If you have baseline numbers for MovieLens/Netflix could you please share it with us ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-63745222
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23642/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27528238
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -171,18 +175,62 @@ object MovieLensALS {
     
         println(s"Test RMSE = $rmse.")
     
    +    if (params.validateRecommendation) {
    +      val (map, users) = computeRankingMetrics(model,
    +        training, test, numMovies.toInt)
    +      println(s"Test users $users MAP $map")
    +    }
    +    
         sc.stop()
       }
     
       /** Compute RMSE (Root Mean Squared Error). */
       def computeRmse(model: MatrixFactorizationModel, data: RDD[Rating], implicitPrefs: Boolean) = {
    -
    -    def mapPredictedRating(r: Double) = if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    -
         val predictions: RDD[Rating] = model.predict(data.map(x => (x.user, x.product)))
    -    val predictionsAndRatings = predictions.map{ x =>
    -      ((x.user, x.product), mapPredictedRating(x.rating))
    +    val predictionsAndRatings = predictions.map { x =>
    +      ((x.user, x.product), mapPredictedRating(x.rating, implicitPrefs))
         }.join(data.map(x => ((x.user, x.product), x.rating))).values
         math.sqrt(predictionsAndRatings.map(x => (x._1 - x._2) * (x._1 - x._2)).mean())
       }
    +
    +  def mapPredictedRating(r: Double, implicitPrefs: Boolean) = {
    +    if (implicitPrefs) math.max(math.min(r, 1.0), 0.0)
    +    else r
    +  }
    +
    +  /**
    +   * Compute MAP (Mean Average Precision) statistics for top N product Recommendation
    +   */
    +  def computeRankingMetrics(model: MatrixFactorizationModel,
    +    train: RDD[Rating], test: RDD[Rating], n: Int) = {
    +
    +    val ord = Ordering.by[(Int, Double), Double](x => x._2)
    +
    +    val testUserLabels = test.map {
    +      x => (x.user, (x.product, x.rating))
    +    }.groupByKey.map {
    +      case (userId, products) =>
    +        val sortedProducts = products.toArray.sorted(ord.reverse)
    +        (userId, sortedProducts.map { _._1 })
    --- End diff --
    
    fixed


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-61750042
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22909/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-62670877
  
    More performance tests undergoing for internal datasets where the cartesian code is really slow (due to groupByKey)...For MovieLens there is no substantial difference...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27529231
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -17,14 +17,20 @@
     
     package org.apache.spark.mllib.recommendation
     
    -import java.lang.{Integer => JavaInteger}
    -
    -import org.jblas.DoubleMatrix
    +import java.lang.{ Integer => JavaInteger }
     
     import org.apache.spark.SparkContext._
    -import org.apache.spark.api.java.{JavaPairRDD, JavaRDD}
    +import org.apache.spark.api.java.{ JavaPairRDD, JavaRDD }
    --- End diff --
    
    cleaned


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27528071
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -171,18 +175,62 @@ object MovieLensALS {
     
         println(s"Test RMSE = $rmse.")
     
    +    if (params.validateRecommendation) {
    +      val (map, users) = computeRankingMetrics(model,
    +        training, test, numMovies.toInt)
    +      println(s"Test users $users MAP $map")
    +    }
    +    
         sc.stop()
       }
     
       /** Compute RMSE (Root Mean Squared Error). */
       def computeRmse(model: MatrixFactorizationModel, data: RDD[Rating], implicitPrefs: Boolean) = {
    -
    -    def mapPredictedRating(r: Double) = if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    -
         val predictions: RDD[Rating] = model.predict(data.map(x => (x.user, x.product)))
    -    val predictionsAndRatings = predictions.map{ x =>
    -      ((x.user, x.product), mapPredictedRating(x.rating))
    +    val predictionsAndRatings = predictions.map { x =>
    +      ((x.user, x.product), mapPredictedRating(x.rating, implicitPrefs))
         }.join(data.map(x => ((x.user, x.product), x.rating))).values
         math.sqrt(predictionsAndRatings.map(x => (x._1 - x._2) * (x._1 - x._2)).mean())
       }
    +
    +  def mapPredictedRating(r: Double, implicitPrefs: Boolean) = {
    +    if (implicitPrefs) math.max(math.min(r, 1.0), 0.0)
    +    else r
    +  }
    +
    +  /**
    +   * Compute MAP (Mean Average Precision) statistics for top N product Recommendation
    +   */
    +  def computeRankingMetrics(model: MatrixFactorizationModel,
    --- End diff --
    
    followed the indentation from current code


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r24955626
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -39,14 +39,15 @@ import org.apache.spark.rdd.RDD
     object MovieLensALS {
     
       case class Params(
    -      input: String = null,
    -      kryo: Boolean = false,
    -      numIterations: Int = 20,
    -      lambda: Double = 1.0,
    -      rank: Int = 10,
    -      numUserBlocks: Int = -1,
    -      numProductBlocks: Int = -1,
    -      implicitPrefs: Boolean = false) extends AbstractParams[Params]
    +    input: String = null,
    --- End diff --
    
    Please fix the indentation.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27528198
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -171,18 +175,62 @@ object MovieLensALS {
     
         println(s"Test RMSE = $rmse.")
     
    +    if (params.validateRecommendation) {
    +      val (map, users) = computeRankingMetrics(model,
    +        training, test, numMovies.toInt)
    +      println(s"Test users $users MAP $map")
    +    }
    +    
         sc.stop()
       }
     
       /** Compute RMSE (Root Mean Squared Error). */
       def computeRmse(model: MatrixFactorizationModel, data: RDD[Rating], implicitPrefs: Boolean) = {
    -
    -    def mapPredictedRating(r: Double) = if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    -
         val predictions: RDD[Rating] = model.predict(data.map(x => (x.user, x.product)))
    -    val predictionsAndRatings = predictions.map{ x =>
    -      ((x.user, x.product), mapPredictedRating(x.rating))
    +    val predictionsAndRatings = predictions.map { x =>
    +      ((x.user, x.product), mapPredictedRating(x.rating, implicitPrefs))
         }.join(data.map(x => ((x.user, x.product), x.rating))).values
         math.sqrt(predictionsAndRatings.map(x => (x._1 - x._2) * (x._1 - x._2)).mean())
       }
    +
    +  def mapPredictedRating(r: Double, implicitPrefs: Boolean) = {
    +    if (implicitPrefs) math.max(math.min(r, 1.0), 0.0)
    +    else r
    +  }
    +
    +  /**
    +   * Compute MAP (Mean Average Precision) statistics for top N product Recommendation
    +   */
    +  def computeRankingMetrics(model: MatrixFactorizationModel,
    +    train: RDD[Rating], test: RDD[Rating], n: Int) = {
    +
    +    val ord = Ordering.by[(Int, Double), Double](x => x._2)
    +
    +    val testUserLabels = test.map {
    +      x => (x.user, (x.product, x.rating))
    +    }.groupByKey.map {
    --- End diff --
    
    fixed


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r24955671
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -35,33 +41,33 @@ import org.apache.spark.rdd.RDD
      *                        and the features computed for this product.
      */
     class MatrixFactorizationModel private[mllib] (
    -    val rank: Int,
    -    val userFeatures: RDD[(Int, Array[Double])],
    -    val productFeatures: RDD[(Int, Array[Double])]) extends Serializable {
    +  val rank: Int,
    --- End diff --
    
    Do not change indentation. The previous 4-space indentation is correct.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-61750040
  
      [Test build #22909 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22909/consoleFull) for   PR 3098 at commit [`9b3951f`](https://github.com/apache/spark/commit/9b3951f558e5673eb475c575f14876421b5a3abc).
     * This patch **fails Scala style tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-62248281
  
      [Test build #23091 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23091/consoleFull) for   PR 3098 at commit [`f38a1b5`](https://github.com/apache/spark/commit/f38a1b59e27907f2aa9bd732c5f9147b738d3a0f).
     * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-88347022
  
    @mengxr could you please do another pass....I might have missed the JavaRDD compatibility issue but fixed rest of your comments...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-88312097
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29518/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r24955662
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -171,18 +175,62 @@ object MovieLensALS {
     
         println(s"Test RMSE = $rmse.")
     
    +    if (params.validateRecommendation) {
    +      val (map, users) = computeRankingMetrics(model,
    +        training, test, numMovies.toInt)
    +      println(s"Test users $users MAP $map")
    +    }
    +    
         sc.stop()
       }
     
       /** Compute RMSE (Root Mean Squared Error). */
       def computeRmse(model: MatrixFactorizationModel, data: RDD[Rating], implicitPrefs: Boolean) = {
    -
    -    def mapPredictedRating(r: Double) = if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    -
         val predictions: RDD[Rating] = model.predict(data.map(x => (x.user, x.product)))
    -    val predictionsAndRatings = predictions.map{ x =>
    -      ((x.user, x.product), mapPredictedRating(x.rating))
    +    val predictionsAndRatings = predictions.map { x =>
    +      ((x.user, x.product), mapPredictedRating(x.rating, implicitPrefs))
         }.join(data.map(x => ((x.user, x.product), x.rating))).values
         math.sqrt(predictionsAndRatings.map(x => (x._1 - x._2) * (x._1 - x._2)).mean())
       }
    +
    +  def mapPredictedRating(r: Double, implicitPrefs: Boolean) = {
    +    if (implicitPrefs) math.max(math.min(r, 1.0), 0.0)
    +    else r
    +  }
    +
    +  /**
    +   * Compute MAP (Mean Average Precision) statistics for top N product Recommendation
    +   */
    +  def computeRankingMetrics(model: MatrixFactorizationModel,
    +    train: RDD[Rating], test: RDD[Rating], n: Int) = {
    +
    +    val ord = Ordering.by[(Int, Double), Double](x => x._2)
    +
    +    val testUserLabels = test.map {
    +      x => (x.user, (x.product, x.rating))
    +    }.groupByKey.map {
    +      case (userId, products) =>
    +        val sortedProducts = products.toArray.sorted(ord.reverse)
    +        (userId, sortedProducts.map { _._1 })
    +    }
    +
    +    val trainUserLabels = train.map {
    +      x => (x.user, x.product)
    +    }.groupByKey.map {
    +      case (userId, products) => (userId, products.toArray)
    +    }
    +    
    +    val rankings = model.recommendProductsForUsers(n).join(trainUserLabels).map {
    +      case (userId, (pred, train)) => {
    +        val predictedProducts = pred.map { _.product }
    --- End diff --
    
    `pred.map(_.product)`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r20050415
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -145,11 +154,14 @@ object MovieLensALS {
            * the confidence. The error is the difference between prediction and either 1 or 0,
            * depending on whether r is positive or negative.
            */
    -      splits(1).map(x => Rating(x.user, x.product, if (x.rating > 0) 1.0 else 0.0))
    +      testSplit.map(x => Rating(x.user, x.product, if (x.rating > 0) 1.0 else 0.0))
         } else {
    -      splits(1)
    +      testSplit
         }.cache()
     
    +    training.cache
    --- End diff --
    
    `cache -> cache()`, which is an action.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-63979463
  
    More details about the API added and experiments are on the JIRA


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r20617279
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -18,14 +18,14 @@
     package org.apache.spark.examples.mllib
     
     import scala.collection.mutable
    -
    -import org.apache.log4j.{Level, Logger}
    +import org.apache.log4j.{ Level, Logger }
    --- End diff --
    
    Fixed the codiing syle...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-61818259
  
    @srowen Let me use the rmse from multiclass metrics and update it....


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-88311846
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29517/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-88346990
  
    I reran the map computation on MovieLens with varying ranks:
    
    Example run:
    ./bin/spark-submit --master spark://TUSCA09LMLVT00C.local:7077 --class org.apache.spark.examples.mllib.MovieLensALS --jars ~/.m2/repository/com/github/scopt/scopt_2.10/3.2.0/scopt_2.10-3.2.0.jar --total-executor-cores 4 --executor-memory 4g --driver-memory 1g ./examples/target/spark-examples_2.10-1.4.0-SNAPSHOT.jar --lambda 0.065 --metrics map ~/datasets/ml-1m/ratings.dat  
    
    rank = default
    
    Got 1000209 ratings from 6040 users on 3706 movies.                             
    Training: 800187, test: 200022.
    Test users 6035 MAP 0.03499984595868497                                         
    
    rank = 25
    
    Got 1000209 ratings from 6040 users on 3706 movies.                             
    Training: 799385, test: 200824.
    Test users 6034 MAP 0.042580954047373255                                        
    
    rank = 50
    
    Got 1000209 ratings from 6040 users on 3706 movies.                             
    Training: 800289, test: 199920.
    Test users 6036 MAP 0.048958415806933275
    
    rank = 100
    
    Got 1000209 ratings from 6040 users on 3706 movies.                             
    Training: 801148, test: 199061.
    Test users 6038 MAP 0.05503487765882986
    
    The numbers are consistent with my runs before.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-61896399
  
      [Test build #22953 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22953/consoleFull) for   PR 3098 at commit [`cd3ab31`](https://github.com/apache/spark/commit/cd3ab31cb9b244bae2b45396a6269ed1dc59151b).
     * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-88361341
  
      [Test build #29521 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29521/consoleFull) for   PR 3098 at commit [`3a0c4eb`](https://github.com/apache/spark/commit/3a0c4eb7f81ee0845f4945d395f6652c965f941b).
     * This patch **passes all tests**.
     * This patch merges cleanly.
     * This patch adds the following public classes _(experimental)_:
      * `  case class FeatureTopK(feature: Vector, topK: Int)`
      * `class UDFRegistration(object):`
    
     * This patch does not change any dependencies.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27707375
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -138,14 +141,122 @@ class MatrixFactorizationModel(
       }
     
       private def recommend(
    -      recommendToFeatures: Array[Double],
    -      recommendableFeatures: RDD[(Int, Array[Double])],
    -      num: Int): Array[(Int, Double)] = {
    -    val scored = recommendableFeatures.map { case (id,features) =>
    -      (id, blas.ddot(features.length, recommendToFeatures, 1, features, 1))
    +    recommendToFeatures: Array[Double],
    +    recommendableFeatures: RDD[(Int, Array[Double])],
    +    num: Int): Array[(Int, Double)] = {
    +    val recommendToVector = Vectors.dense(recommendToFeatures)
    +    val scored = recommendableFeatures.map {
    +      case (id, features) =>
    +        (id, BLAS.dot(recommendToVector, Vectors.dense(features)))
         }
         scored.top(num)(Ordering.by(_._2))
       }
    +
    +  /**
    +   * Recommends topK products for all users
    +   *
    +   * @param num how many products to return for every user.
    +   * @return [(Int, Array[Rating])] objects, where every tuple contains a userID and an array of
    +   * rating objects which contains the same userId, recommended productID and a "score" in the
    +   * rating field. Semantics of score is same as recommendProducts API
    +   */
    +  def recommendProductsForUsers(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = userFeatures.map { x => (x._1, num) }
    +    recommendProductsForUsers(topK)
    +  }
    +
    +  /**
    +   * Recommends topK users for all products
    +   *
    +   * @param num how many users to return for every product.
    +   * @return [(Int, Array[Rating])] objects, where every tuple contains a productID and an array
    +   * of rating objects which contains the recommended userId, same productID and a "score" in the
    +   * rating field. Semantics of score is same as recommendUsers API
    +   */
    +  def recommendUsersForProducts(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = productFeatures.map { x => (x._1, num) }
    +    recommendUsersForProducts(topK)
    +  }
    +
    +  val ord = Ordering.by[Rating, Double](x => x.rating)
    +  case class FeatureTopK(feature: Vector, topK: Int)
    --- End diff --
    
    This is not necessary if we use a global `num`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-96403986
  
    was very last few weeks...update it in next few days...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27529347
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -35,33 +41,33 @@ import org.apache.spark.rdd.RDD
      *                        and the features computed for this product.
      */
     class MatrixFactorizationModel private[mllib] (
    -    val rank: Int,
    -    val userFeatures: RDD[(Int, Array[Double])],
    -    val productFeatures: RDD[(Int, Array[Double])]) extends Serializable {
    +  val rank: Int,
    +  val userFeatures: RDD[(Int, Array[Double])],
    +  val productFeatures: RDD[(Int, Array[Double])]) extends Serializable {
       /** Predict the rating of one user for one product. */
       def predict(user: Int, product: Int): Double = {
    -    val userVector = new DoubleMatrix(userFeatures.lookup(user).head)
    -    val productVector = new DoubleMatrix(productFeatures.lookup(product).head)
    -    userVector.dot(productVector)
    +    val userVector = Vectors.dense(userFeatures.lookup(user).head)
    --- End diff --
    
    I cleaned netlib.ddot to BLAS.dot...they will be same for these cases


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r24955678
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -103,13 +109,106 @@ class MatrixFactorizationModel private[mllib] (
         recommend(productFeatures.lookup(product).head, userFeatures, num)
           .map(t => Rating(t._1, product, t._2))
     
    +  /**
    +   * Recommends topK users/products.
    +   *
    +   * @param num how many users to return. The number returned may be less than this.
    --- End diff --
    
    This block doesn't associate with any method. Please move these docs to the corresponding methods' JavaDoc.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r24955648
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -171,18 +175,62 @@ object MovieLensALS {
     
         println(s"Test RMSE = $rmse.")
     
    +    if (params.validateRecommendation) {
    +      val (map, users) = computeRankingMetrics(model,
    +        training, test, numMovies.toInt)
    +      println(s"Test users $users MAP $map")
    +    }
    +    
         sc.stop()
       }
     
       /** Compute RMSE (Root Mean Squared Error). */
       def computeRmse(model: MatrixFactorizationModel, data: RDD[Rating], implicitPrefs: Boolean) = {
    -
    -    def mapPredictedRating(r: Double) = if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    -
         val predictions: RDD[Rating] = model.predict(data.map(x => (x.user, x.product)))
    -    val predictionsAndRatings = predictions.map{ x =>
    -      ((x.user, x.product), mapPredictedRating(x.rating))
    +    val predictionsAndRatings = predictions.map { x =>
    +      ((x.user, x.product), mapPredictedRating(x.rating, implicitPrefs))
         }.join(data.map(x => ((x.user, x.product), x.rating))).values
         math.sqrt(predictionsAndRatings.map(x => (x._1 - x._2) * (x._1 - x._2)).mean())
       }
    +
    +  def mapPredictedRating(r: Double, implicitPrefs: Boolean) = {
    +    if (implicitPrefs) math.max(math.min(r, 1.0), 0.0)
    --- End diff --
    
    Fix style. Use `if .. else ..` without `{ .. }` only if it can fit into a single line. (https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide#SparkCodeStyleGuide-CurlyBraces)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r24955650
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -171,18 +175,62 @@ object MovieLensALS {
     
         println(s"Test RMSE = $rmse.")
     
    +    if (params.validateRecommendation) {
    +      val (map, users) = computeRankingMetrics(model,
    +        training, test, numMovies.toInt)
    +      println(s"Test users $users MAP $map")
    +    }
    +    
         sc.stop()
       }
     
       /** Compute RMSE (Root Mean Squared Error). */
       def computeRmse(model: MatrixFactorizationModel, data: RDD[Rating], implicitPrefs: Boolean) = {
    -
    -    def mapPredictedRating(r: Double) = if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    -
         val predictions: RDD[Rating] = model.predict(data.map(x => (x.user, x.product)))
    -    val predictionsAndRatings = predictions.map{ x =>
    -      ((x.user, x.product), mapPredictedRating(x.rating))
    +    val predictionsAndRatings = predictions.map { x =>
    +      ((x.user, x.product), mapPredictedRating(x.rating, implicitPrefs))
         }.join(data.map(x => ((x.user, x.product), x.rating))).values
         math.sqrt(predictionsAndRatings.map(x => (x._1 - x._2) * (x._1 - x._2)).mean())
       }
    +
    +  def mapPredictedRating(r: Double, implicitPrefs: Boolean) = {
    +    if (implicitPrefs) math.max(math.min(r, 1.0), 0.0)
    +    else r
    +  }
    +
    +  /**
    +   * Compute MAP (Mean Average Precision) statistics for top N product Recommendation
    +   */
    +  def computeRankingMetrics(model: MatrixFactorizationModel,
    --- End diff --
    
    fix indentation: https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide#SparkCodeStyleGuide-Indentation


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27528959
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -171,18 +175,62 @@ object MovieLensALS {
     
         println(s"Test RMSE = $rmse.")
     
    +    if (params.validateRecommendation) {
    +      val (map, users) = computeRankingMetrics(model,
    +        training, test, numMovies.toInt)
    +      println(s"Test users $users MAP $map")
    +    }
    +    
         sc.stop()
       }
     
       /** Compute RMSE (Root Mean Squared Error). */
       def computeRmse(model: MatrixFactorizationModel, data: RDD[Rating], implicitPrefs: Boolean) = {
    -
    -    def mapPredictedRating(r: Double) = if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    -
         val predictions: RDD[Rating] = model.predict(data.map(x => (x.user, x.product)))
    -    val predictionsAndRatings = predictions.map{ x =>
    -      ((x.user, x.product), mapPredictedRating(x.rating))
    +    val predictionsAndRatings = predictions.map { x =>
    +      ((x.user, x.product), mapPredictedRating(x.rating, implicitPrefs))
         }.join(data.map(x => ((x.user, x.product), x.rating))).values
         math.sqrt(predictionsAndRatings.map(x => (x._1 - x._2) * (x._1 - x._2)).mean())
       }
    +
    +  def mapPredictedRating(r: Double, implicitPrefs: Boolean) = {
    +    if (implicitPrefs) math.max(math.min(r, 1.0), 0.0)
    +    else r
    +  }
    +
    +  /**
    +   * Compute MAP (Mean Average Precision) statistics for top N product Recommendation
    +   */
    +  def computeRankingMetrics(model: MatrixFactorizationModel,
    +    train: RDD[Rating], test: RDD[Rating], n: Int) = {
    +
    +    val ord = Ordering.by[(Int, Double), Double](x => x._2)
    +
    +    val testUserLabels = test.map {
    +      x => (x.user, (x.product, x.rating))
    +    }.groupByKey.map {
    +      case (userId, products) =>
    +        val sortedProducts = products.toArray.sorted(ord.reverse)
    +        (userId, sortedProducts.map { _._1 })
    +    }
    +
    +    val trainUserLabels = train.map {
    +      x => (x.user, x.product)
    +    }.groupByKey.map {
    +      case (userId, products) => (userId, products.toArray)
    --- End diff --
    
    merged


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r24955659
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -171,18 +175,62 @@ object MovieLensALS {
     
         println(s"Test RMSE = $rmse.")
     
    +    if (params.validateRecommendation) {
    +      val (map, users) = computeRankingMetrics(model,
    +        training, test, numMovies.toInt)
    +      println(s"Test users $users MAP $map")
    +    }
    +    
         sc.stop()
       }
     
       /** Compute RMSE (Root Mean Squared Error). */
       def computeRmse(model: MatrixFactorizationModel, data: RDD[Rating], implicitPrefs: Boolean) = {
    -
    -    def mapPredictedRating(r: Double) = if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    -
         val predictions: RDD[Rating] = model.predict(data.map(x => (x.user, x.product)))
    -    val predictionsAndRatings = predictions.map{ x =>
    -      ((x.user, x.product), mapPredictedRating(x.rating))
    +    val predictionsAndRatings = predictions.map { x =>
    +      ((x.user, x.product), mapPredictedRating(x.rating, implicitPrefs))
         }.join(data.map(x => ((x.user, x.product), x.rating))).values
         math.sqrt(predictionsAndRatings.map(x => (x._1 - x._2) * (x._1 - x._2)).mean())
       }
    +
    +  def mapPredictedRating(r: Double, implicitPrefs: Boolean) = {
    +    if (implicitPrefs) math.max(math.min(r, 1.0), 0.0)
    +    else r
    +  }
    +
    +  /**
    +   * Compute MAP (Mean Average Precision) statistics for top N product Recommendation
    +   */
    +  def computeRankingMetrics(model: MatrixFactorizationModel,
    +    train: RDD[Rating], test: RDD[Rating], n: Int) = {
    +
    +    val ord = Ordering.by[(Int, Double), Double](x => x._2)
    +
    +    val testUserLabels = test.map {
    +      x => (x.user, (x.product, x.rating))
    +    }.groupByKey.map {
    +      case (userId, products) =>
    +        val sortedProducts = products.toArray.sorted(ord.reverse)
    +        (userId, sortedProducts.map { _._1 })
    +    }
    +
    +    val trainUserLabels = train.map {
    +      x => (x.user, x.product)
    +    }.groupByKey.map {
    +      case (userId, products) => (userId, products.toArray)
    --- End diff --
    
    merge `case .. =>` to the line above


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r24955680
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -103,13 +109,106 @@ class MatrixFactorizationModel private[mllib] (
         recommend(productFeatures.lookup(product).head, userFeatures, num)
           .map(t => Rating(t._1, product, t._2))
     
    +  /**
    +   * Recommends topK users/products.
    +   *
    +   * @param num how many users to return. The number returned may be less than this.
    +   * @return [Array[Rating]] objects, each of which contains a userID, the given productID and a
    +   *  "score" in the rating field. Each represents one recommended user, and they are sorted
    +   *  by score, decreasing. The first returned is the one predicted to be most strongly
    +   *  recommended to the product. The score is an opaque value that indicates how strongly
    +   *  recommended the user is.
    +   */
    +
    +  /**
    +   * Recommend topK products for all users
    +   */
    +  def recommendProductsForUsers(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = userFeatures.map { x => (x._1, num) }
    +    recommendProductsForUsers(topK)
    +  }
    +
    +  /**
    +   * Recommend topK users for all products
    +   */
    +  def recommendUsersForProducts(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = productFeatures.map { x => (x._1, num) }
    +    recommendUsersForProducts(topK)
    +  }
    +
    +  val ord = Ordering.by[Rating, Double](x => x.rating)
    --- End diff --
    
    These become public members. Please move them to the companion object and mark them package private.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27528991
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -171,18 +175,62 @@ object MovieLensALS {
     
         println(s"Test RMSE = $rmse.")
     
    +    if (params.validateRecommendation) {
    +      val (map, users) = computeRankingMetrics(model,
    +        training, test, numMovies.toInt)
    +      println(s"Test users $users MAP $map")
    +    }
    +    
         sc.stop()
       }
     
       /** Compute RMSE (Root Mean Squared Error). */
       def computeRmse(model: MatrixFactorizationModel, data: RDD[Rating], implicitPrefs: Boolean) = {
    -
    -    def mapPredictedRating(r: Double) = if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    -
         val predictions: RDD[Rating] = model.predict(data.map(x => (x.user, x.product)))
    -    val predictionsAndRatings = predictions.map{ x =>
    -      ((x.user, x.product), mapPredictedRating(x.rating))
    +    val predictionsAndRatings = predictions.map { x =>
    +      ((x.user, x.product), mapPredictedRating(x.rating, implicitPrefs))
         }.join(data.map(x => ((x.user, x.product), x.rating))).values
         math.sqrt(predictionsAndRatings.map(x => (x._1 - x._2) * (x._1 - x._2)).mean())
       }
    +
    +  def mapPredictedRating(r: Double, implicitPrefs: Boolean) = {
    +    if (implicitPrefs) math.max(math.min(r, 1.0), 0.0)
    +    else r
    +  }
    +
    +  /**
    +   * Compute MAP (Mean Average Precision) statistics for top N product Recommendation
    +   */
    +  def computeRankingMetrics(model: MatrixFactorizationModel,
    +    train: RDD[Rating], test: RDD[Rating], n: Int) = {
    +
    +    val ord = Ordering.by[(Int, Double), Double](x => x._2)
    +
    +    val testUserLabels = test.map {
    +      x => (x.user, (x.product, x.rating))
    +    }.groupByKey.map {
    +      case (userId, products) =>
    +        val sortedProducts = products.toArray.sorted(ord.reverse)
    +        (userId, sortedProducts.map { _._1 })
    +    }
    +
    +    val trainUserLabels = train.map {
    +      x => (x.user, x.product)
    +    }.groupByKey.map {
    +      case (userId, products) => (userId, products.toArray)
    +    }
    +    
    +    val rankings = model.recommendProductsForUsers(n).join(trainUserLabels).map {
    +      case (userId, (pred, train)) => {
    +        val predictedProducts = pred.map { _.product }
    --- End diff --
    
    done


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27707360
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -167,23 +169,66 @@ object MovieLensALS {
           .setProductBlocks(params.numProductBlocks)
           .run(training)
     
    -    val rmse = computeRmse(model, test, params.implicitPrefs)
    -
    -    println(s"Test RMSE = $rmse.")
    +    params.metrics match {
    +      case "rmse" =>
    +        val rmse = computeRmse(model, test, params.implicitPrefs)
    +        println(s"Test RMSE = $rmse")
    +      case "map" =>
    +        val (map, users) = computeRankingMetrics(model, training, test, numMovies.toInt)
    +        println(s"Test users $users MAP $map")
    +      case _ => println(s"Metrics not defined, options are rmse/map")
    +    }
     
         sc.stop()
       }
     
       /** Compute RMSE (Root Mean Squared Error). */
    -  def computeRmse(model: MatrixFactorizationModel, data: RDD[Rating], implicitPrefs: Boolean)
    -    : Double = {
    -
    -    def mapPredictedRating(r: Double) = if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    -
    +  def computeRmse(
    +    model: MatrixFactorizationModel,
    +    data: RDD[Rating],
    +    implicitPrefs: Boolean) : Double = {
         val predictions: RDD[Rating] = model.predict(data.map(x => (x.user, x.product)))
    -    val predictionsAndRatings = predictions.map{ x =>
    -      ((x.user, x.product), mapPredictedRating(x.rating))
    +    val predictionsAndRatings = predictions.map { x =>
    +      ((x.user, x.product), mapPredictedRating(x.rating, implicitPrefs))
         }.join(data.map(x => ((x.user, x.product), x.rating))).values
         math.sqrt(predictionsAndRatings.map(x => (x._1 - x._2) * (x._1 - x._2)).mean())
       }
    +
    +  def mapPredictedRating(r: Double, implicitPrefs: Boolean) = {
    +    if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    +  }
    +  
    +  /** Compute MAP (Mean Average Precision) statistics for top N product Recommendation */
    +  def computeRankingMetrics(
    +    model: MatrixFactorizationModel,
    +    train: RDD[Rating],
    +    test: RDD[Rating],
    +    n: Int) : (Double, Long) = {
    +    val ord = Ordering.by[(Int, Double), Double](x => x._2)
    +
    +    val testUserLabels = test.map {
    +      x => (x.user, (x.product, x.rating))
    +    }.groupByKey().map {
    +      case (userId, products) =>
    +        val sortedProducts = products.toArray.sorted(ord.reverse)
    +        (userId, sortedProducts.map(_._1))
    +    }
    +
    +    val trainUserLabels = train.map {
    +      x => (x.user, x.product)
    +    }.groupByKey().map{case (userId, products) => (userId, products.toArray)}
    --- End diff --
    
    `.groupByKey().mapValues(_.toArray)`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-61905697
  
      [Test build #22963 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22963/consoleFull) for   PR 3098 at commit [`9fa063e`](https://github.com/apache/spark/commit/9fa063e1eb172d68248e03797a54acc738543592).
     * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-89729377
  
    I meant MAP...what's the MAP on netflix dataset you have seen before and with what lambda ? I am running MAP experiments with various factorization formulations including loglikelihood loss with normalization constraints...also how do you define MAP for implicit feedback (binary dataset, click is 1 and no click is 0) ? In the label set every rating is 1.0 and so there is no ranking defined as such...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-61913884
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22963/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27712646
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -138,14 +141,122 @@ class MatrixFactorizationModel(
       }
     
       private def recommend(
    -      recommendToFeatures: Array[Double],
    -      recommendableFeatures: RDD[(Int, Array[Double])],
    -      num: Int): Array[(Int, Double)] = {
    -    val scored = recommendableFeatures.map { case (id,features) =>
    -      (id, blas.ddot(features.length, recommendToFeatures, 1, features, 1))
    +    recommendToFeatures: Array[Double],
    +    recommendableFeatures: RDD[(Int, Array[Double])],
    +    num: Int): Array[(Int, Double)] = {
    +    val recommendToVector = Vectors.dense(recommendToFeatures)
    +    val scored = recommendableFeatures.map {
    +      case (id, features) =>
    +        (id, BLAS.dot(recommendToVector, Vectors.dense(features)))
         }
         scored.top(num)(Ordering.by(_._2))
       }
    +
    +  /**
    +   * Recommends topK products for all users
    +   *
    +   * @param num how many products to return for every user.
    +   * @return [(Int, Array[Rating])] objects, where every tuple contains a userID and an array of
    +   * rating objects which contains the same userId, recommended productID and a "score" in the
    +   * rating field. Semantics of score is same as recommendProducts API
    +   */
    +  def recommendProductsForUsers(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = userFeatures.map { x => (x._1, num) }
    --- End diff --
    
    For cross validation we use variable num internally but for final recommendation global num is fine...I thought having a topK rdd satisfies both use-cases...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r24955652
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -171,18 +175,62 @@ object MovieLensALS {
     
         println(s"Test RMSE = $rmse.")
     
    +    if (params.validateRecommendation) {
    +      val (map, users) = computeRankingMetrics(model,
    +        training, test, numMovies.toInt)
    +      println(s"Test users $users MAP $map")
    +    }
    +    
         sc.stop()
       }
     
       /** Compute RMSE (Root Mean Squared Error). */
       def computeRmse(model: MatrixFactorizationModel, data: RDD[Rating], implicitPrefs: Boolean) = {
    -
    -    def mapPredictedRating(r: Double) = if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    -
         val predictions: RDD[Rating] = model.predict(data.map(x => (x.user, x.product)))
    -    val predictionsAndRatings = predictions.map{ x =>
    -      ((x.user, x.product), mapPredictedRating(x.rating))
    +    val predictionsAndRatings = predictions.map { x =>
    +      ((x.user, x.product), mapPredictedRating(x.rating, implicitPrefs))
         }.join(data.map(x => ((x.user, x.product), x.rating))).values
         math.sqrt(predictionsAndRatings.map(x => (x._1 - x._2) * (x._1 - x._2)).mean())
       }
    +
    +  def mapPredictedRating(r: Double, implicitPrefs: Boolean) = {
    +    if (implicitPrefs) math.max(math.min(r, 1.0), 0.0)
    +    else r
    +  }
    +
    +  /**
    +   * Compute MAP (Mean Average Precision) statistics for top N product Recommendation
    +   */
    +  def computeRankingMetrics(model: MatrixFactorizationModel,
    +    train: RDD[Rating], test: RDD[Rating], n: Int) = {
    --- End diff --
    
    add return type


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-88311999
  
      [Test build #29518 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29518/consoleFull) for   PR 3098 at commit [`98fa424`](https://github.com/apache/spark/commit/98fa4243dc6041290bdde51e1e899a8be7576470).
     * This patch **does not merge cleanly**.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27535273
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -103,13 +109,106 @@ class MatrixFactorizationModel private[mllib] (
         recommend(productFeatures.lookup(product).head, userFeatures, num)
           .map(t => Rating(t._1, product, t._2))
     
    +  /**
    +   * Recommends topK users/products.
    +   *
    +   * @param num how many users to return. The number returned may be less than this.
    +   * @return [Array[Rating]] objects, each of which contains a userID, the given productID and a
    +   *  "score" in the rating field. Each represents one recommended user, and they are sorted
    +   *  by score, decreasing. The first returned is the one predicted to be most strongly
    +   *  recommended to the product. The score is an opaque value that indicates how strongly
    +   *  recommended the user is.
    +   */
    +
    +  /**
    +   * Recommend topK products for all users
    +   */
    +  def recommendProductsForUsers(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = userFeatures.map { x => (x._1, num) }
    +    recommendProductsForUsers(topK)
    +  }
    +
    +  /**
    +   * Recommend topK users for all products
    +   */
    +  def recommendUsersForProducts(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = productFeatures.map { x => (x._1, num) }
    +    recommendUsersForProducts(topK)
    +  }
    +
    +  val ord = Ordering.by[Rating, Double](x => x.rating)
    +  case class FeatureTopK(feature: Vector, topK: Int)
    +
    +  /**
    +   * Recommend topK products for users in userTopK RDD
    --- End diff --
    
    documented the public batch prediction APIs


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27707386
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -138,14 +141,122 @@ class MatrixFactorizationModel(
       }
     
       private def recommend(
    -      recommendToFeatures: Array[Double],
    -      recommendableFeatures: RDD[(Int, Array[Double])],
    -      num: Int): Array[(Int, Double)] = {
    -    val scored = recommendableFeatures.map { case (id,features) =>
    -      (id, blas.ddot(features.length, recommendToFeatures, 1, features, 1))
    +    recommendToFeatures: Array[Double],
    +    recommendableFeatures: RDD[(Int, Array[Double])],
    +    num: Int): Array[(Int, Double)] = {
    +    val recommendToVector = Vectors.dense(recommendToFeatures)
    +    val scored = recommendableFeatures.map {
    +      case (id, features) =>
    +        (id, BLAS.dot(recommendToVector, Vectors.dense(features)))
         }
         scored.top(num)(Ordering.by(_._2))
       }
    +
    +  /**
    +   * Recommends topK products for all users
    +   *
    +   * @param num how many products to return for every user.
    +   * @return [(Int, Array[Rating])] objects, where every tuple contains a userID and an array of
    +   * rating objects which contains the same userId, recommended productID and a "score" in the
    +   * rating field. Semantics of score is same as recommendProducts API
    +   */
    +  def recommendProductsForUsers(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = userFeatures.map { x => (x._1, num) }
    +    recommendProductsForUsers(topK)
    +  }
    +
    +  /**
    +   * Recommends topK users for all products
    +   *
    +   * @param num how many users to return for every product.
    +   * @return [(Int, Array[Rating])] objects, where every tuple contains a productID and an array
    +   * of rating objects which contains the recommended userId, same productID and a "score" in the
    +   * rating field. Semantics of score is same as recommendUsers API
    +   */
    +  def recommendUsersForProducts(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = productFeatures.map { x => (x._1, num) }
    +    recommendUsersForProducts(topK)
    +  }
    +
    +  val ord = Ordering.by[Rating, Double](x => x.rating)
    +  case class FeatureTopK(feature: Vector, topK: Int)
    +
    +  /**
    +   * Recommend topK products for users in userTopK RDD
    +   *
    +   * @param userTopK how many products to return for every user in userTopK RDD.
    +   * @return [(Int, Array[Rating])] objects, where every tuple contains a userID and an array
    +   * of rating objects which contains the same userId, recommended productID and a "score" in the
    +   * rating field. Semantics of score is same as recommendProducts API
    +   */
    +  def recommendProductsForUsers(
    +    userTopK: RDD[(Int, Int)]): RDD[(Int, Array[Rating])] = {
    +    val userFeaturesTopK = userFeatures.join(userTopK).map {
    +      case (userId, (userFeature, topK)) =>
    +        (userId, FeatureTopK(Vectors.dense(userFeature), topK))
    +    }
    +
    +    // TO DO: Do a mini-batch on productFeatures.collect if the dimension of rows are big
    +    val productVectors = productFeatures.map {
    +      x => (x._1, Vectors.dense(x._2))
    +    }.collect
    +
    +    // TO DO: User BLAS dgemm
    +    userFeaturesTopK.map {
    +      case (userId, userFeatureTopK) => {
    +        val predictions = productVectors.map {
    +          case (productId, productVector) =>
    +            Rating(userId, productId,
    +              BLAS.dot(userFeatureTopK.feature, productVector))
    +        }
    +        (userId, Utils.takeOrdered(predictions.iterator,
    +          userFeatureTopK.topK)(ord.reverse).toArray)
    +      }
    +    }
    +  }
    +
    +  /**
    +   * Recommends topK users for all products in productTopK RDD
    +   *
    +   * @param productTopK how many users to return for every product in productTopK RDD
    +   * @return [(Int, Array[Rating])] objects, where every tuple contains a productID and an array
    +   * of Rating objects which contains the recommended userId, same productID and a "score" in the
    +   * rating field. Semantics of score is same as recommendUsers API
    +   */
    +  def recommendUsersForProducts(
    +    productTopK: RDD[(Int, Int)]): RDD[(Int, Array[Rating])] = {
    +    val blocks = userFeatures.partitions.size / 2
    +
    +    // TO DO: Do a mini-batch on productFeatures.collect if the dimension of rows are big
    +    val productVectors = productFeatures.join(productTopK).map {
    +      case (productId, (productFeature, topK)) =>
    +        (productId, FeatureTopK(Vectors.dense(productFeature), topK))
    +    }.collect()
    +
    +    // TO DO: Use BLAS dgemm
    +    userFeatures.mapPartitions { items =>
    +      val predictions = productVectors.map {
    +        x => (x._1, new BoundedPriorityQueue[Rating](x._2.topK)(ord.reverse))
    +      }.toMap
    +      while (items.hasNext) {
    +        val (userId, userFeature) = items.next
    +        val userVector = Vectors.dense(userFeature)
    +        for (i <- 0 until productVectors.length) {
    +          val (productId, productFeatureTopK) = productVectors(i)
    +          val predicted = Rating(userId, productId,
    +            BLAS.dot(userVector, productFeatureTopK.feature))
    +          predictions(productId) ++= Iterator.single(predicted)
    --- End diff --
    
    `+= predicted`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-62675921
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23246/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-89697236
  
    @srowen For netflix dataset what's the MAP you have seen before...I started experiments on Netflix dataset...lambda is 0.065 for netflix as well right ? For MovieLens 0.065 works well...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27533769
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -103,13 +109,106 @@ class MatrixFactorizationModel private[mllib] (
         recommend(productFeatures.lookup(product).head, userFeatures, num)
           .map(t => Rating(t._1, product, t._2))
     
    +  /**
    +   * Recommends topK users/products.
    +   *
    +   * @param num how many users to return. The number returned may be less than this.
    +   * @return [Array[Rating]] objects, each of which contains a userID, the given productID and a
    +   *  "score" in the rating field. Each represents one recommended user, and they are sorted
    +   *  by score, decreasing. The first returned is the one predicted to be most strongly
    +   *  recommended to the product. The score is an opaque value that indicates how strongly
    +   *  recommended the user is.
    +   */
    +
    +  /**
    +   * Recommend topK products for all users
    +   */
    +  def recommendProductsForUsers(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = userFeatures.map { x => (x._1, num) }
    +    recommendProductsForUsers(topK)
    +  }
    +
    +  /**
    +   * Recommend topK users for all products
    +   */
    +  def recommendUsersForProducts(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = productFeatures.map { x => (x._1, num) }
    +    recommendUsersForProducts(topK)
    +  }
    +
    +  val ord = Ordering.by[Rating, Double](x => x.rating)
    --- End diff --
    
    I am bit confused...recommendProducts is also a public member but that's not in companion object...recommendProductsForUsers is also very similar API right ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-61913879
  
      [Test build #22963 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22963/consoleFull) for   PR 3098 at commit [`9fa063e`](https://github.com/apache/spark/commit/9fa063e1eb172d68248e03797a54acc738543592).
     * This patch **passes all tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-62675915
  
      [Test build #23246 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23246/consoleFull) for   PR 3098 at commit [`d144f57`](https://github.com/apache/spark/commit/d144f57a58c9424365f1242f90961386c016641e).
     * This patch **passes all tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-62248133
  
    @coderxiang @MLnick updated the PR with the steps mentioned above...I experimented with recommendProducts API but to support map-reduce validation, decided to use cartesian of user and product feature RDD...
    
    It seems for MAP computation we can do further optimization and we don't need rank of n predicted products but we can send the rank of the products that are only in testSet...It needs change to RatingMatrix as well and most likely should be a separate PR...Not sure if this optimization holds for prec@k and ndcg@k as well...
    
    Here are the results on MovieLens 1M dataset:
    
    ./bin/spark-submit --master spark://localhost:7077 --jars ~/.m2/repository/com/github/scopt/scopt_2.10/3.2.0/scopt_2.10-3.2.0.jar --total-executor-cores 4 --executor-memory 4g --driver-memory 1g --class org.apache.spark.examples.mllib.MovieLensALS ./examples/target/spark-examples_2.10-1.2.0-SNAPSHOT.jar --kryo --lambda 0.065 --validateRecommendation 1.0 hdfs://localhost:8020/sandbox/movielens/
    
    Got 1000209 ratings from 6040 users on 3706 movies.
    Training: 800392, test: 199817.
    Test RMSE = 0.8492844823210867.
    k 20 prec@k 0.05337860218615437
    k 40 prec@k 0.05016561775422324
    k 60 prec@k 0.04697471568952191
    k 80 prec@k 0.044406260351109623
    k 100 prec@k 0.04242630009937065
    k 120 prec@k 0.040632935850723195
    k 140 prec@k 0.0390656792693891
    k 160 prec@k 0.037725654190129174
    k 180 prec@k 0.03652699569393839
    k 200 prec@k 0.035531632991056654
    Test user MAP = 0.038054284339533735.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-62670490
  
      [Test build #23246 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23246/consoleFull) for   PR 3098 at commit [`d144f57`](https://github.com/apache/spark/commit/d144f57a58c9424365f1242f90961386c016641e).
     * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-63722805
  
      [Test build #23632 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23632/consoleFull) for   PR 3098 at commit [`7163a5c`](https://github.com/apache/spark/commit/7163a5c21b394d8bd89694a9f08aa1b446c71956).
     * This patch merges cleanly.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-89706247
  
    @coderxiang @mengxr If I have a dataset with implicit (click or 0) then MAP is not that well defined right since in label set everything is 1.0 and so there is no ordering defined....should we add a rank independent metric for implicit datasets ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-89728639
  
    @debasish83 do you mean RMSE? it is well-defined but not very useful. MAP is the useful metric. I think that only a rank-dependent metric makes sense.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-61896572
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22953/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-63738974
  
    **[Test build #23632 timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23632/consoleFull)**     for PR 3098 at commit [`7163a5c`](https://github.com/apache/spark/commit/7163a5c21b394d8bd89694a9f08aa1b446c71956)     after a configured wait of `120m`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by MLnick <gi...@git.apache.org>.
Github user MLnick commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-62509181
  
    @debasish83 @srowen yes, these MAP@k numbers look similar to what I computed on some old MovieLens experiments (after adjusting to use implicit prefs and the `-2.5` rating adjustment). 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-61910380
  
      [Test build #22957 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22957/consoleFull) for   PR 3098 at commit [`4bbae0f`](https://github.com/apache/spark/commit/4bbae0f248ca8747b47ecf852d5aba19c9b39dab).
     * This patch **passes all tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r20050275
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -39,14 +39,15 @@ import org.apache.spark.rdd.RDD
     object MovieLensALS {
     
       case class Params(
    -      input: String = null,
    -      kryo: Boolean = false,
    -      numIterations: Int = 20,
    -      lambda: Double = 1.0,
    -      rank: Int = 10,
    -      numUserBlocks: Int = -1,
    -      numProductBlocks: Int = -1,
    -      implicitPrefs: Boolean = false) extends AbstractParams[Params]
    +    input: String = null,
    --- End diff --
    
    This is a class declaration, where we use 4-space indentation


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by debasish83 <gi...@git.apache.org>.
Github user debasish83 commented on the pull request:

    https://github.com/apache/spark/pull/3098#issuecomment-61896998
  
    @srowen RMSE for both user and product view are similar and so I did not call multiclass metric..also as per discussion before with @mengxr we are still not sure to consider recommendation as a multiclass classification.
    
    I refactored the code to compute both user and product MAP separately. I still use the old way to compute RMSE
    
    Here are the results on MovieLens:
    
    ./bin/spark-submit --master spark://tusca09lmlvt00c.uswin.ad.vzwcorp.com:7077 --jars /Users/v606014/.m2/repository/com/github/scopt/scopt_2.10/3.2.0/scopt_2.10-3.2.0.jar --total-executor-cores 4 --executor-memory 4g --driver-memory 1g --class org.apache.spark.examples.mllib.MovieLensALS ./examples/target/spark-examples_2.10-1.2.0-SNAPSHOT.jar --kryo --lambda 0.065 hdfs://localhost:8020/sandbox/movielens/
    2014-11-05 14:42:31.287 java[11464:1903] Unable to load realm mapping info from SCDynamicStore
    14/11/05 14:42:31 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    Got 1000209 ratings from 6040 users on 3706 movies.
    Training: 799710, test: 200499.
    Test RMSE = 0.8974937349724504 user MAP = 7.432139222366133 product MAP = 12.203619639224904.
    
    Should I generate the testSet based on more specific random sampling ? Basically right now I am sampling the whole RDD for 20% test but I can also do user / product based sampling...did that help you before in Oryx ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231: Add RankingMetrics to exam...

Posted by coderxiang <gi...@git.apache.org>.
Github user coderxiang commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r19980455
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -165,22 +169,60 @@ object MovieLensALS {
           .setProductBlocks(params.numProductBlocks)
           .run(training)
     
    -    val rmse = computeRmse(model, test, params.implicitPrefs)
    -
    -    println(s"Test RMSE = $rmse.")
    -
    +    val (rmse, userMap, productMap) = 
    +      computeRecommendationMetrics(model, test, params.implicitPrefs)
    +    
    +    println(s"Test RMSE = $rmse user MAP = $userMap product MAP = $productMap.")
    +    
         sc.stop()
       }
    -
    -  /** Compute RMSE (Root Mean Squared Error). */
    -  def computeRmse(model: MatrixFactorizationModel, data: RDD[Rating], implicitPrefs: Boolean) = {
    -
    -    def mapPredictedRating(r: Double) = if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    -
    +  
    +  /**  
    +   * Threshold for predictions are at 0.5
    +   */
    +  def mapPredictedRating(r: Double, implicitPrefs: Boolean) = {
    +    if (implicitPrefs) math.max(math.min(r, 1.0), 0.0)
    +    else math.max(round(r), 0.0)
    +  }
    +  
    +  /**  
    +   * Compute MAP (Mean Average Precision) statistics
    +   */
    +  def computeMap(predictedAndLabels: RDD[(Int, (Double, Double))]) = {
    +     val ranking = predictedAndLabels.groupByKey.map {
    +      case (user, entries) => {
    +        val predictionValues = entries.toArray
    --- End diff --
    
    Please kindly point out if I understand incorrectly. Is the `entries` here be the value of ratings? The `predictedAndLables` is supposed to (predicted ID, true ID), but not (predicted rating, true rating). I think this might also be the reason you get an MAP value great than 1.0, which is by definition not possible...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r24955657
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -171,18 +175,62 @@ object MovieLensALS {
     
         println(s"Test RMSE = $rmse.")
     
    +    if (params.validateRecommendation) {
    +      val (map, users) = computeRankingMetrics(model,
    +        training, test, numMovies.toInt)
    +      println(s"Test users $users MAP $map")
    +    }
    +    
         sc.stop()
       }
     
       /** Compute RMSE (Root Mean Squared Error). */
       def computeRmse(model: MatrixFactorizationModel, data: RDD[Rating], implicitPrefs: Boolean) = {
    -
    -    def mapPredictedRating(r: Double) = if (implicitPrefs) math.max(math.min(r, 1.0), 0.0) else r
    -
         val predictions: RDD[Rating] = model.predict(data.map(x => (x.user, x.product)))
    -    val predictionsAndRatings = predictions.map{ x =>
    -      ((x.user, x.product), mapPredictedRating(x.rating))
    +    val predictionsAndRatings = predictions.map { x =>
    +      ((x.user, x.product), mapPredictedRating(x.rating, implicitPrefs))
         }.join(data.map(x => ((x.user, x.product), x.rating))).values
         math.sqrt(predictionsAndRatings.map(x => (x._1 - x._2) * (x._1 - x._2)).mean())
       }
    +
    +  def mapPredictedRating(r: Double, implicitPrefs: Boolean) = {
    +    if (implicitPrefs) math.max(math.min(r, 1.0), 0.0)
    +    else r
    +  }
    +
    +  /**
    +   * Compute MAP (Mean Average Precision) statistics for top N product Recommendation
    +   */
    +  def computeRankingMetrics(model: MatrixFactorizationModel,
    +    train: RDD[Rating], test: RDD[Rating], n: Int) = {
    +
    +    val ord = Ordering.by[(Int, Double), Double](x => x._2)
    +
    +    val testUserLabels = test.map {
    +      x => (x.user, (x.product, x.rating))
    +    }.groupByKey.map {
    +      case (userId, products) =>
    +        val sortedProducts = products.toArray.sorted(ord.reverse)
    +        (userId, sortedProducts.map { _._1 })
    --- End diff --
    
    `.map(_._1)`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r24955630
  
    --- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala ---
    @@ -74,6 +75,9 @@ object MovieLensALS {
           opt[Unit]("implicitPrefs")
             .text("use implicit preference")
             .action((_, c) => c.copy(implicitPrefs = true))
    +      opt[Unit]("validateRecommendation")
    --- End diff --
    
    We can switch the evaluation metric between RMSE and MAP via a `--metrics` option. This is more specific than `validateRecommendation`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request: [MLLIB] SPARK-4231, SPARK-3066: Add RankingMet...

Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:

    https://github.com/apache/spark/pull/3098#discussion_r27707380
  
    --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala ---
    @@ -138,14 +141,122 @@ class MatrixFactorizationModel(
       }
     
       private def recommend(
    -      recommendToFeatures: Array[Double],
    -      recommendableFeatures: RDD[(Int, Array[Double])],
    -      num: Int): Array[(Int, Double)] = {
    -    val scored = recommendableFeatures.map { case (id,features) =>
    -      (id, blas.ddot(features.length, recommendToFeatures, 1, features, 1))
    +    recommendToFeatures: Array[Double],
    +    recommendableFeatures: RDD[(Int, Array[Double])],
    +    num: Int): Array[(Int, Double)] = {
    +    val recommendToVector = Vectors.dense(recommendToFeatures)
    +    val scored = recommendableFeatures.map {
    +      case (id, features) =>
    +        (id, BLAS.dot(recommendToVector, Vectors.dense(features)))
         }
         scored.top(num)(Ordering.by(_._2))
       }
    +
    +  /**
    +   * Recommends topK products for all users
    +   *
    +   * @param num how many products to return for every user.
    +   * @return [(Int, Array[Rating])] objects, where every tuple contains a userID and an array of
    +   * rating objects which contains the same userId, recommended productID and a "score" in the
    +   * rating field. Semantics of score is same as recommendProducts API
    +   */
    +  def recommendProductsForUsers(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = userFeatures.map { x => (x._1, num) }
    +    recommendProductsForUsers(topK)
    +  }
    +
    +  /**
    +   * Recommends topK users for all products
    +   *
    +   * @param num how many users to return for every product.
    +   * @return [(Int, Array[Rating])] objects, where every tuple contains a productID and an array
    +   * of rating objects which contains the recommended userId, same productID and a "score" in the
    +   * rating field. Semantics of score is same as recommendUsers API
    +   */
    +  def recommendUsersForProducts(num: Int): RDD[(Int, Array[Rating])] = {
    +    val topK = productFeatures.map { x => (x._1, num) }
    +    recommendUsersForProducts(topK)
    +  }
    +
    +  val ord = Ordering.by[Rating, Double](x => x.rating)
    +  case class FeatureTopK(feature: Vector, topK: Int)
    +
    +  /**
    +   * Recommend topK products for users in userTopK RDD
    +   *
    +   * @param userTopK how many products to return for every user in userTopK RDD.
    +   * @return [(Int, Array[Rating])] objects, where every tuple contains a userID and an array
    +   * of rating objects which contains the same userId, recommended productID and a "score" in the
    +   * rating field. Semantics of score is same as recommendProducts API
    +   */
    +  def recommendProductsForUsers(
    +    userTopK: RDD[(Int, Int)]): RDD[(Int, Array[Rating])] = {
    +    val userFeaturesTopK = userFeatures.join(userTopK).map {
    +      case (userId, (userFeature, topK)) =>
    +        (userId, FeatureTopK(Vectors.dense(userFeature), topK))
    +    }
    +
    +    // TO DO: Do a mini-batch on productFeatures.collect if the dimension of rows are big
    +    val productVectors = productFeatures.map {
    +      x => (x._1, Vectors.dense(x._2))
    +    }.collect
    +
    +    // TO DO: User BLAS dgemm
    +    userFeaturesTopK.map {
    +      case (userId, userFeatureTopK) => {
    +        val predictions = productVectors.map {
    +          case (productId, productVector) =>
    +            Rating(userId, productId,
    +              BLAS.dot(userFeatureTopK.feature, productVector))
    +        }
    +        (userId, Utils.takeOrdered(predictions.iterator,
    +          userFeatureTopK.topK)(ord.reverse).toArray)
    --- End diff --
    
    Use the `topByKey` method to simplify the code.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org