You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Andrew Palumbo (JIRA)" <ji...@apache.org> on 2016/12/19 06:07:59 UTC

[jira] [Commented] (MAHOUT-1786) Make classes implements Serializable for Spark 1.5+

    [ https://issues.apache.org/jira/browse/MAHOUT-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15760270#comment-15760270 ] 

Andrew Palumbo commented on MAHOUT-1786:
----------------------------------------

This (MAHOUT-1786) may be a part of the issue  of the broken broadcast issues. 

> Make classes implements Serializable for Spark 1.5+
> ---------------------------------------------------
>
>                 Key: MAHOUT-1786
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-1786
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Math
>    Affects Versions: 0.11.0
>            Reporter: Michel Lemay
>            Priority: Minor
>              Labels: performance
>
> Spark 1.5 comes with a new very efficient serializer that uses code generation.  It is twice as fast as kryo.  When using mahout, we have to set KryoSerializer because some classes aren't serializable otherwise.  
> I suggest to declare Math classes as "implements Serializable" where needed.  For instance, to use coocurence package in spark 1.5, we had to modify AbstractMatrix, AbstractVector, DenseVector and SparseRowMatrix to make it work without Kryo.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)