You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by nevillelyh <gi...@git.apache.org> on 2014/05/30 21:26:15 UTC
[GitHub] spark pull request: [SPARK-1977][MLLIB] use immutable BitSet in AL...
GitHub user nevillelyh opened a pull request:
https://github.com/apache/spark/pull/925
[SPARK-1977][MLLIB] use immutable BitSet in ALS
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nevillelyh/spark branch-1.0
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/925.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #925
----
commit 72fec8ed0a647faacd7e9530252a1ec0aa898281
Author: Neville Li <ne...@spotify.com>
Date: 2014-05-30T19:00:36Z
[SPARK-1977][MLLIB] use immutable BitSet in ALS
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1977][MLLIB] use immutable BitSet in AL...
Posted by aaronlin <gi...@git.apache.org>.
Github user aaronlin commented on the pull request:
https://github.com/apache/spark/pull/925#issuecomment-63159256
Spark-ec2.py gets the official release of spark, so even if i build spark with the registration of bitset and rating class manually, I cannot use it when deploying ec2 cluster. Can you give me some hints how to cope with the problem while using spark-ec2.py? Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-1977][MLLIB] use immutable BitSet in AL...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/925#issuecomment-44718196
Maybe we should just make sure the mutable bitsets are registered instead of doing the conversion?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1977][MLLIB] use immutable BitSet in AL...
Posted by nevillelyh <gi...@git.apache.org>.
Github user nevillelyh commented on the pull request:
https://github.com/apache/spark/pull/925#issuecomment-44848781
Added support to chill. Will bump version after next chill release.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1977][MLLIB] use immutable BitSet in AL...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/925#issuecomment-44690150
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1977][MLLIB] use immutable BitSet in AL...
Posted by nevillelyh <gi...@git.apache.org>.
Github user nevillelyh closed the pull request at:
https://github.com/apache/spark/pull/925
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1977][MLLIB] use immutable BitSet in AL...
Posted by aaronlin <gi...@git.apache.org>.
Github user aaronlin commented on the pull request:
https://github.com/apache/spark/pull/925#issuecomment-63015981
twitter/chill#185 fixed in chill v0.4.0, but spark still depends on chill v0.3.6 in maven.
http://mvnrepository.com/artifact/org.apache.spark/spark-core_2.10/1.1.0
Can anyone help to fix it?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-1977][MLLIB] use immutable BitSet in AL...
Posted by nevillelyh <gi...@git.apache.org>.
Github user nevillelyh commented on the pull request:
https://github.com/apache/spark/pull/925#issuecomment-44750919
Sure that would also work. I made a PR to chill as well.
https://github.com/twitter/chill/pull/185
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1977][MLLIB] use immutable BitSet in AL...
Posted by nevillelyh <gi...@git.apache.org>.
Github user nevillelyh commented on the pull request:
https://github.com/apache/spark/pull/925#issuecomment-48223111
Closing this and will fix MovieLensALS & chill dependency once next version is released.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1977][MLLIB] use immutable BitSet in AL...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/925#issuecomment-63128071
@aaronlin Changing dependency requires more testing, we don't usually do that in a maintenance release. Could you register the BitSet and Rating class manually?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-1977][MLLIB] use immutable BitSet in AL...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/925#issuecomment-63202927
The registration could be done through configuration. For example, there is an `ALSRegistrator` under `examples/` and you can enable it through configuration without changing any Spark code:
~~~
bin/pyspark --master local-cluster[2,1,512] --conf 'spark.kryo.registrator=org.apache.spark.examples.mllib.MovieLensALS$ALSRegistrator' --jars lib/spark-examples-1.1.0-hadoop2.4.0.jar
~~~
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-1977][MLLIB] use immutable BitSet in AL...
Posted by aaronlin <gi...@git.apache.org>.
Github user aaronlin commented on the pull request:
https://github.com/apache/spark/pull/925#issuecomment-63325849
I tried your solution and run the program successfully on v1.1.0. Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-1977][MLLIB] use immutable BitSet in AL...
Posted by aaronlin <gi...@git.apache.org>.
Github user aaronlin commented on the pull request:
https://github.com/apache/spark/pull/925#issuecomment-63203111
Thanks for your reply. I will try this solution. Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org