You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Seth Hendrickson (JIRA)" <ji...@apache.org> on 2016/07/06 22:17:11 UTC
[jira] [Commented] (SPARK-16404) LeastSquaresAggregator in Linear
Regression serializes unnecessary data
[ https://issues.apache.org/jira/browse/SPARK-16404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365227#comment-15365227 ]
Seth Hendrickson commented on SPARK-16404:
------------------------------------------
cc [~dbtsai] I looked in to using the @transient tag, but this prevents the coefficients from being serialized and broadcast to the executors at all, resulting in a {{NullPointerException}}. I am not sure of a way around this. I can submit a patch utilizing the same strategy as in LoR later this week.
> LeastSquaresAggregator in Linear Regression serializes unnecessary data
> -----------------------------------------------------------------------
>
> Key: SPARK-16404
> URL: https://issues.apache.org/jira/browse/SPARK-16404
> Project: Spark
> Issue Type: Improvement
> Components: ML
> Reporter: Seth Hendrickson
>
> This is basically the same issue as [SPARK-16008|https://issues.apache.org/jira/browse/SPARK-16008], but for linear regression, where {{coefficients}} and {{featuresStd}} are unnecessarily serialized between stages.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org