You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by smartkiwi <gi...@git.apache.org> on 2015/09/10 17:43:48 UTC
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
GitHub user smartkiwi opened a pull request:
https://github.com/apache/spark/pull/8700
[SPARK-10535] Sync up API for matrix factorization model between Scala and PySpark
Support for recommendUsersForProducts and recommendProductsForUsers in matrix factorization model for PySpark
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/smartkiwi/spark SPARK-10535_
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/8700.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #8700
----
commit 68f2a09a87495fe2d860fa9164a9a590bdf522d8
Author: Vladimir Vladimirov <vl...@magnetic.com>
Date: 2015-09-10T13:11:47Z
[SPARK-10535] Support for recommendUsersForProducts and recommendProductsForUsers in matrix factorization model for PySpark
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-142099277
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42762/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/8700#discussion_r39292338
--- Diff: python/pyspark/mllib/recommendation.py ---
@@ -85,6 +85,12 @@ class MatrixFactorizationModel(JavaModelWrapper, JavaSaveable, JavaLoader):
>>> len(latents) == 4
True
+ >>> products_for_users = model.recommendProductsForUsers(1).collect()
+ ...
--- End diff --
The output is deterministic. So we should do the following:
~~~python
>>> model.recommendProductsForUsers(1).collect()
// put expected output here
~~~
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by smartkiwi <gi...@git.apache.org>.
Github user smartkiwi commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-139398333
I've added these methods doc string
Acually it is not that useful as it is now - as it returns "scala.Tuple2" strings - and not as in scala (Int, Array[Rating])
model.recommendUsersForProducts(1).collect()
returns
[{u'__class__': u'scala.Tuple2'}, {u'__class__': u'scala.Tuple2'}]
So there is no way to access user id and product record
@davies could you please point me how to update scala SerDe/ and python py4j related code to unwrap scala.Tuple2 into python tuple of int and Rating objects?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/8700#discussion_r39553069
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/MatrixFactorizationModelWrapper.scala ---
@@ -42,4 +42,15 @@ private[python] class MatrixFactorizationModelWrapper(model: MatrixFactorization
case (product, feature) => (product, Vectors.dense(feature))
}.asInstanceOf[RDD[(Any, Any)]])
}
+
+ def wrappedRecommendProductsForUsers(num: Int): RDD[Array[Any]] = {
+ SerDe.fromTuple2RDD(recommendProductsForUsers(num).asInstanceOf[RDD[(Any, Any)]])
+ }
+
+ def wrappedRecommendUsersForProducts(num: Int): RDD[Array[Any]] = {
+ SerDe.fromTuple2RDD(recommendUsersForProducts(num).asInstanceOf[RDD[(Any, Any)]]
+ )
--- End diff --
move `)` to the line above
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by smartkiwi <gi...@git.apache.org>.
Github user smartkiwi commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-139653064
Ok
Please see this update - I've added methods for converting Tuple to Array into MatrixFactorizationModelWrapper
Could you please review this pull request again
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-144180554
[Test build #1826 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1826/consoleFull) for PR 8700 at commit [`56d3321`](https://github.com/apache/spark/commit/56d33214d324fc158b29ff3b08f7c409f19433a1).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/8700#discussion_r39292344
--- Diff: python/pyspark/mllib/recommendation.py ---
@@ -156,6 +162,18 @@ def recommendProducts(self, user, num):
"""
return list(self.call("recommendProducts", user, num))
+ def recommendProductsForUsers(self, num):
+ """
+ Recommends topK products for all users.
+ """
+ return self.call("recommendProductsForUsers", num)
--- End diff --
It won't work because it will return `scala.Tuple2`. We need to add a helper on the Scala side to make it work. Using `DataFrame` should work.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by smartkiwi <gi...@git.apache.org>.
Github user smartkiwi commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-140479983
@davies ping
Can somebody verify this? Jenkins have't started tests for this pull requests
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-142095875
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-139287240
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/8700
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-140498376
[Test build #1756 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1756/console) for PR 8700 at commit [`bc7dc46`](https://github.com/apache/spark/commit/bc7dc4600ad7aa9ac0a7c3ce6c1ae2697dbab329).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by smartkiwi <gi...@git.apache.org>.
Github user smartkiwi commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-142152718
Please test this again - there were some github connection issue
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-142184249
[Test build #1783 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1783/console) for PR 8700 at commit [`969e4c3`](https://github.com/apache/spark/commit/969e4c343de6922dd672a8a46e82f8a338b129a7).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-142180737
[Test build #1783 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1783/consoleFull) for PR 8700 at commit [`969e4c3`](https://github.com/apache/spark/commit/969e4c343de6922dd672a8a46e82f8a338b129a7).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-142095847
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-142094998
test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by smartkiwi <gi...@git.apache.org>.
Github user smartkiwi commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-142287774
This fixes pyspark tests.
Please test this again.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-142099272
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by davies <gi...@git.apache.org>.
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-146987818
LGTM, merging this into master, thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/8700#discussion_r39553213
--- Diff: python/pyspark/mllib/recommendation.py ---
@@ -85,6 +85,16 @@ class MatrixFactorizationModel(JavaModelWrapper, JavaSaveable, JavaLoader):
>>> len(latents) == 4
True
+ >>> products_for_users = model.recommendProductsForUsers(1).collect()
+ >>> len(products_for_users) == 2
--- End diff --
Should be
~~~python
>>> len(products_for_users)
2
~~~
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by davies <gi...@git.apache.org>.
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-139377677
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by smartkiwi <gi...@git.apache.org>.
Github user smartkiwi commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-141793243
Jenkins please test again
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/8700#discussion_r39553076
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/MatrixFactorizationModelWrapper.scala ---
@@ -42,4 +42,15 @@ private[python] class MatrixFactorizationModelWrapper(model: MatrixFactorization
case (product, feature) => (product, Vectors.dense(feature))
}.asInstanceOf[RDD[(Any, Any)]])
}
+
+ def wrappedRecommendProductsForUsers(num: Int): RDD[Array[Any]] = {
+ SerDe.fromTuple2RDD(recommendProductsForUsers(num).asInstanceOf[RDD[(Any, Any)]])
+ }
+
+ def wrappedRecommendUsersForProducts(num: Int): RDD[Array[Any]] = {
+ SerDe.fromTuple2RDD(recommendUsersForProducts(num).asInstanceOf[RDD[(Any, Any)]]
+ )
+ }
+
+
--- End diff --
remove extra blank lines
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by davies <gi...@git.apache.org>.
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-139403576
Oh, I missed that part.
You could add a wrapper in Scala (PythonMLLibAPI) to convert RDD[Tuple] into RDD[Array[Any]], and call the wrapper from Python.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by davies <gi...@git.apache.org>.
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-139377664
OK to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/8700#discussion_r39553301
--- Diff: python/pyspark/mllib/recommendation.py ---
@@ -156,6 +166,18 @@ def recommendProducts(self, user, num):
"""
return list(self.call("recommendProducts", user, num))
+ def recommendProductsForUsers(self, num):
+ """
+ Recommends topK products for all users.
--- End diff --
document `num`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-144215760
[Test build #1826 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1826/console) for PR 8700 at commit [`56d3321`](https://github.com/apache/spark/commit/56d33214d324fc158b29ff3b08f7c409f19433a1).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by smartkiwi <gi...@git.apache.org>.
Github user smartkiwi commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-142093257
Ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-139602565
@smartkiwi Sorry, my comments duplicate @davies 's ...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by smartkiwi <gi...@git.apache.org>.
Github user smartkiwi commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-146665724
Test passes. Any feedback on this? Or can this be merged?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-140489529
[Test build #1756 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1756/consoleFull) for PR 8700 at commit [`bc7dc46`](https://github.com/apache/spark/commit/bc7dc4600ad7aa9ac0a7c3ce6c1ae2697dbab329).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10535] Sync up API for matrix factoriza...
Posted by smartkiwi <gi...@git.apache.org>.
Github user smartkiwi commented on the pull request:
https://github.com/apache/spark/pull/8700#issuecomment-139605916
@mengxr, @davies thanks for a hints
I'm rather new with Scala, and very interested into learning it further.
I've found the code with Python wrapper used for other recommendation methods
https://github.com/smartkiwi/spark/blob/SPARK-10535/mllib/src/main/scala/org/apache/spark/mllib/api/python/MatrixFactorizationModelWrapper.scala#L34-L43
I'm working on using similar approach for the methods I'm adding
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org