You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2015/11/06 16:37:27 UTC
[jira] [Commented] (MAHOUT-1786) Make classes implements
Serializable for Spark 1.5+
[ https://issues.apache.org/jira/browse/MAHOUT-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14993824#comment-14993824 ]
ASF GitHub Bot commented on MAHOUT-1786:
----------------------------------------
GitHub user michellemay opened a pull request:
https://github.com/apache/mahout/pull/174
MAHOUT-1786: Make classes implements Serializable for Spark 1.5+
Add some "implements Serializable" for Apache Spark 1.5+
There might be other classes that would benefit from the same modification.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/michellemay/mahout fix-mahout-1786
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/mahout/pull/174.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #174
----
commit a0aad6a716d001c5bbe1ee08e8865dbb83b6f1e4
Author: michellemay <ml...@gmail.com>
Date: 2015-11-06T15:34:04Z
implements Serializable
----
> Make classes implements Serializable for Spark 1.5+
> ---------------------------------------------------
>
> Key: MAHOUT-1786
> URL: https://issues.apache.org/jira/browse/MAHOUT-1786
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Affects Versions: 0.11.0
> Reporter: Michel Lemay
> Priority: Minor
> Labels: performance
>
> Spark 1.5 comes with a new very efficient serializer that uses code generation. It is twice as fast as kryo. When using mahout, we have to set KryoSerializer because some classes aren't serializable otherwise.
> I suggest to declare Math classes as "implements Serializable" where needed. For instance, to use coocurence package in spark 1.5, we had to modify AbstractMatrix, AbstractVector, DenseVector and SparseRowMatrix to make it work without Kryo.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)