You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by mengxr <gi...@git.apache.org> on 2014/03/18 05:53:12 UTC
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
GitHub user mengxr opened a pull request:
https://github.com/apache/spark/pull/165
[SPARK-1266] persist factors in implicit ALS
In implicit ALS computation, the user or product factor is used twice in each iteration. Caching can certainly help accelerate the computation. I saw the running time decreased by ~70% for implicit ALS on the movielens data.
I also made the following changes:
1. Change `YtYb` type from `Broadcast[Option[DoubleMatrix]]` to `Option[Broadcast[DoubleMatrix]]`, so we don't need to broadcast None in explicit computation.
2. Mark methods `computeYtY`, `unblockFactors`, `updateBlock`, and `updateFeatures private`. Users do not need those methods.
3. Materialize the final matrix factors before returning the model. It allows us to clean up other cached RDDs before returning the model. I do not have a better solution here, so I use `RDD.count()`.
JIRA: https://spark-project.atlassian.net/browse/SPARK-1266
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mengxr/spark als
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/165.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #165
----
commit 63862d63a8ae06e21a52918c0097a7f1c843ee5d
Author: Xiangrui Meng <me...@databricks.com>
Date: 2014-03-17T23:20:24Z
persist factors in implicit ALS
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/165#discussion_r10719159
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/ALS.scala ---
@@ -187,21 +189,39 @@ class ALS private (
}
}
- for (iter <- 1 to iterations) {
- // perform ALS update
- logInfo("Re-computing I given U (Iteration %d/%d)".format(iter, iterations))
- // YtY / XtX is an Option[DoubleMatrix] and is only required for the implicit feedback model
- val YtY = computeYtY(users)
- val YtYb = ratings.context.broadcast(YtY)
- products = updateFeatures(users, userOutLinks, productInLinks, partitioner, rank, lambda,
- alpha, YtYb)
- logInfo("Re-computing U given I (Iteration %d/%d)".format(iter, iterations))
- val XtX = computeYtY(products)
- val XtXb = ratings.context.broadcast(XtX)
- users = updateFeatures(products, productOutLinks, userInLinks, partitioner, rank, lambda,
- alpha, XtXb)
+ if (implicitPrefs) {
+ for (iter <- 1 to iterations) {
+ // perform ALS update
+ logInfo("Re-computing I given U (Iteration %d/%d)".format(iter, iterations))
+ // Persist users because it will be called twice.
+ users.persist()
+ val YtY = Some(sc.broadcast(computeYtY(users)))
+ val previousProducts = products
+ products = updateFeatures(users, userOutLinks, productInLinks, partitioner, rank, lambda,
+ alpha, YtY)
+ previousProducts.unpersist()
+ logInfo("Re-computing U given I (Iteration %d/%d)".format(iter, iterations))
+ products.persist()
+ val XtX = Some(sc.broadcast(computeYtY(products)))
+ val previousUsers = users
+ users = updateFeatures(products, productOutLinks, userInLinks, partitioner, rank, lambda,
+ alpha, XtX)
+ previousUsers.unpersist()
+ }
+ } else {
+ for (iter <- 1 to iterations) {
+ // perform ALS update
+ logInfo("Re-computing I given U (Iteration %d/%d)".format(iter, iterations))
+ products = updateFeatures(users, userOutLinks, productInLinks, partitioner, rank, lambda,
+ alpha, YtY = None)
+ logInfo("Re-computing U given I (Iteration %d/%d)".format(iter, iterations))
+ users = updateFeatures(products, productOutLinks, userInLinks, partitioner, rank, lambda,
+ alpha, YtY = None)
+ }
}
+ products.persist()
--- End diff --
Added a comment.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
Re: [GitHub] spark pull request: [SPARK-1266] persist factors in
implicit ALS
Posted by Debasish Das <de...@gmail.com>.
Matei,
Are there any benchmarks published with respect to Graphlab for the mllib
algorithms that we can look at ?
Thanks.
Deb
On Mon, Mar 17, 2014 at 11:22 PM, mateiz <gi...@git.apache.org> wrote:
> Github user mateiz commented on the pull request:
>
> https://github.com/apache/spark/pull/165#issuecomment-37902896
>
> Both look like good fixes -- I didn't realize before that each RDD
> would be used twice. This will actually make us very competitive with
> GraphLab for example in benchmarks.
>
> One thing we may want to do later is to make the storage level for
> these persists configurable, but it doesn't seem necessary for now. In
> general managing memory use by libraries will be an interesting question.
>
>
> ---
> If your project is set up for it, you can reply to this email and have your
> reply appear on GitHub as well. If your project does not have this feature
> enabled and wishes so, or if the feature is enabled but not working, please
> contact infrastructure at infrastructure@apache.org or file a JIRA ticket
> with INFRA.
> ---
>
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by mateiz <gi...@git.apache.org>.
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/165#issuecomment-37902896
Both look like good fixes -- I didn't realize before that each RDD would be used twice. This will actually make us very competitive with GraphLab for example in benchmarks.
One thing we may want to do later is to make the storage level for these persists configurable, but it doesn't seem necessary for now. In general managing memory use by libraries will be an interesting question.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by mengxr <gi...@git.apache.org>.
GitHub user mengxr reopened a pull request:
https://github.com/apache/spark/pull/165
[SPARK-1266] persist factors in implicit ALS
In implicit ALS computation, the user or product factor is used twice in each iteration. Caching can certainly help accelerate the computation. I saw the running time decreased by ~70% for implicit ALS on the movielens data.
I also made the following changes:
1. Change `YtYb` type from `Broadcast[Option[DoubleMatrix]]` to `Option[Broadcast[DoubleMatrix]]`, so we don't need to broadcast None in explicit computation.
2. Mark methods `computeYtY`, `unblockFactors`, `updateBlock`, and `updateFeatures private`. Users do not need those methods.
3. Materialize the final matrix factors before returning the model. It allows us to clean up other cached RDDs before returning the model. I do not have a better solution here, so I use `RDD.count()`.
JIRA: https://spark-project.atlassian.net/browse/SPARK-1266
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mengxr/spark als
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/165.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #165
----
commit 63862d63a8ae06e21a52918c0097a7f1c843ee5d
Author: Xiangrui Meng <me...@databricks.com>
Date: 2014-03-17T23:20:24Z
persist factors in implicit ALS
commit d3a88aa026b6a3734e6fc515589e604df81913a6
Author: Xiangrui Meng <me...@databricks.com>
Date: 2014-03-18T07:39:29Z
change implicitPrefs match to if ... else ...
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/165#issuecomment-37899573
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/165#issuecomment-37978828
All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13243/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/165#issuecomment-37909963
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/165#issuecomment-37901335
All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13226/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/165#issuecomment-37906540
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/165#discussion_r10718882
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/ALS.scala ---
@@ -187,21 +189,39 @@ class ALS private (
}
}
- for (iter <- 1 to iterations) {
- // perform ALS update
- logInfo("Re-computing I given U (Iteration %d/%d)".format(iter, iterations))
- // YtY / XtX is an Option[DoubleMatrix] and is only required for the implicit feedback model
- val YtY = computeYtY(users)
- val YtYb = ratings.context.broadcast(YtY)
- products = updateFeatures(users, userOutLinks, productInLinks, partitioner, rank, lambda,
- alpha, YtYb)
- logInfo("Re-computing U given I (Iteration %d/%d)".format(iter, iterations))
- val XtX = computeYtY(products)
- val XtXb = ratings.context.broadcast(XtX)
- users = updateFeatures(products, productOutLinks, userInLinks, partitioner, rank, lambda,
- alpha, XtXb)
+ if (implicitPrefs) {
+ for (iter <- 1 to iterations) {
+ // perform ALS update
+ logInfo("Re-computing I given U (Iteration %d/%d)".format(iter, iterations))
+ // Persist users because it will be called twice.
+ users.persist()
+ val YtY = Some(sc.broadcast(computeYtY(users)))
+ val previousProducts = products
+ products = updateFeatures(users, userOutLinks, productInLinks, partitioner, rank, lambda,
+ alpha, YtY)
+ previousProducts.unpersist()
+ logInfo("Re-computing U given I (Iteration %d/%d)".format(iter, iterations))
+ products.persist()
+ val XtX = Some(sc.broadcast(computeYtY(products)))
+ val previousUsers = users
+ users = updateFeatures(products, productOutLinks, userInLinks, partitioner, rank, lambda,
+ alpha, XtX)
+ previousUsers.unpersist()
+ }
+ } else {
+ for (iter <- 1 to iterations) {
+ // perform ALS update
+ logInfo("Re-computing I given U (Iteration %d/%d)".format(iter, iterations))
+ products = updateFeatures(users, userOutLinks, productInLinks, partitioner, rank, lambda,
+ alpha, YtY = None)
+ logInfo("Re-computing U given I (Iteration %d/%d)".format(iter, iterations))
+ users = updateFeatures(products, productOutLinks, userInLinks, partitioner, rank, lambda,
+ alpha, YtY = None)
+ }
}
+ products.persist()
--- End diff --
When the final factors get materialized, both `users` and `products` are called:
~~~
val usersOut = unblockFactors(users, userOutLinks)
val productsOut = unblockFactors(products, productOutLinks)
~~~
However, the last `users` depends on the last `products`. So the last `products` will be used twice.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by mateiz <gi...@git.apache.org>.
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/165#issuecomment-38005375
Looks good to me too. I've merged it, thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/165#issuecomment-45290001
@coderh Please check the discussion about evaluation metrics for implicit preference at https://github.com/apache/spark/pull/597 .
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by coderh <gi...@git.apache.org>.
Github user coderh commented on the pull request:
https://github.com/apache/spark/pull/165#issuecomment-45245775
Hi, @mengxr
You have mentioned that you have tested implicit ALS on the movielens data.
Do you have any evaluation result on that ? I am so curious about the evaluation criterion used in you case and how you interpret the explicit feedback like ratings into implicit case.
Thank you.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/165#issuecomment-37978825
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/165#issuecomment-37899572
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by MLnick <gi...@git.apache.org>.
Github user MLnick commented on the pull request:
https://github.com/apache/spark/pull/165#issuecomment-37905128
This looks good to me +1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by MLnick <gi...@git.apache.org>.
Github user MLnick commented on the pull request:
https://github.com/apache/spark/pull/165#issuecomment-37901597
Yes, I will take a look this evening.
—
Sent from Mailbox for iPhone
On Tue, Mar 18, 2014 at 7:44 AM, UCB AMPLab <no...@github.com>
wrote:
> All automated tests passed.
> Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13226/
> ---
> Reply to this email directly or view it on GitHub:
> https://github.com/apache/spark/pull/165#issuecomment-37901335
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/165#issuecomment-37909964
All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13232/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/165#issuecomment-37899861
@MLnick I saw you implemented the first version of implicit ALS. Do you have time to review this PR? Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/165#issuecomment-37972525
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/165#issuecomment-37902246
Thanks a lot!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/165#issuecomment-37906541
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by mallman <gi...@git.apache.org>.
Github user mallman commented on a diff in the pull request:
https://github.com/apache/spark/pull/165#discussion_r10714229
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/ALS.scala ---
@@ -187,21 +189,39 @@ class ALS private (
}
}
- for (iter <- 1 to iterations) {
- // perform ALS update
- logInfo("Re-computing I given U (Iteration %d/%d)".format(iter, iterations))
- // YtY / XtX is an Option[DoubleMatrix] and is only required for the implicit feedback model
- val YtY = computeYtY(users)
- val YtYb = ratings.context.broadcast(YtY)
- products = updateFeatures(users, userOutLinks, productInLinks, partitioner, rank, lambda,
- alpha, YtYb)
- logInfo("Re-computing U given I (Iteration %d/%d)".format(iter, iterations))
- val XtX = computeYtY(products)
- val XtXb = ratings.context.broadcast(XtX)
- users = updateFeatures(products, productOutLinks, userInLinks, partitioner, rank, lambda,
- alpha, XtXb)
+ if (implicitPrefs) {
+ for (iter <- 1 to iterations) {
+ // perform ALS update
+ logInfo("Re-computing I given U (Iteration %d/%d)".format(iter, iterations))
+ // Persist users because it will be called twice.
+ users.persist()
+ val YtY = Some(sc.broadcast(computeYtY(users)))
+ val previousProducts = products
+ products = updateFeatures(users, userOutLinks, productInLinks, partitioner, rank, lambda,
+ alpha, YtY)
+ previousProducts.unpersist()
+ logInfo("Re-computing U given I (Iteration %d/%d)".format(iter, iterations))
+ products.persist()
+ val XtX = Some(sc.broadcast(computeYtY(products)))
+ val previousUsers = users
+ users = updateFeatures(products, productOutLinks, userInLinks, partitioner, rank, lambda,
+ alpha, XtX)
+ previousUsers.unpersist()
+ }
+ } else {
+ for (iter <- 1 to iterations) {
+ // perform ALS update
+ logInfo("Re-computing I given U (Iteration %d/%d)".format(iter, iterations))
+ products = updateFeatures(users, userOutLinks, productInLinks, partitioner, rank, lambda,
+ alpha, YtY = None)
+ logInfo("Re-computing U given I (Iteration %d/%d)".format(iter, iterations))
+ users = updateFeatures(products, productOutLinks, userInLinks, partitioner, rank, lambda,
+ alpha, YtY = None)
+ }
}
+ products.persist()
--- End diff --
Why is this here?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr closed the pull request at:
https://github.com/apache/spark/pull/165
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/165
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/165#issuecomment-37972529
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/165#issuecomment-37901333
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/165#discussion_r10693017
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/ALS.scala ---
@@ -445,7 +476,7 @@ class ALS private (
// Solve the resulting matrix, which is symmetric and positive-definite
implicitPrefs match {
--- End diff --
Yes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1266] persist factors in implicit ALS
Posted by MLnick <gi...@git.apache.org>.
Github user MLnick commented on a diff in the pull request:
https://github.com/apache/spark/pull/165#discussion_r10692598
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/ALS.scala ---
@@ -445,7 +476,7 @@ class ALS private (
// Solve the resulting matrix, which is symmetric and positive-definite
implicitPrefs match {
--- End diff --
This is a minor style thing but I originally had used a `match` statement all around, and I did change most of them to `if (implicitPrefs) ... else ...`. I seem to have missed this one, perhaps you can make that change while you're at it? :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---