[jira] [Created] (SPARK-18781) Allow MatrixFactorizationModel.predict to skip user/product approximation count

["Eyal Allweil (JIRA)" <ji...@apache.org>]

Eyal Allweil created SPARK-18781:
------------------------------------

             Summary: Allow MatrixFactorizationModel.predict to skip user/product approximation count
                 Key: SPARK-18781
                 URL: https://issues.apache.org/jira/browse/SPARK-18781
             Project: Spark
          Issue Type: Improvement
          Components: MLlib
            Reporter: Eyal Allweil


When [MatrixFactorizationModel.predict|https://spark.apache.org/docs/1.6.1/api/java/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.html#predict(org.apache.spark.rdd.RDD)] is used, it first calculates an approximation count of the users and products in order to determine the most efficient way to proceed. In many cases, the answer to this question is fixed (typically there are more users than products by an order of magnitude) and this check is unnecessary. Adding a parameter to this predict method to allow choosing the implementation (and skipping the check) would be nice.

It would be especially nice in development cycles when you are repeatedly tweaking your model and which pairs you're predicting for and this approximate count represents a meaningful portion of the time you wait for results.

I can provide a pull request with this ability added that preserves the existing behavior.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org