You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean Owen (JIRA)" <ji...@apache.org> on 2016/02/07 10:14:39 UTC
[jira] [Updated] (SPARK-13132) LogisticRegression spends 35% of its
time fetching the standardization parameter
[ https://issues.apache.org/jira/browse/SPARK-13132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen updated SPARK-13132:
------------------------------
Assignee: Gary King
> LogisticRegression spends 35% of its time fetching the standardization parameter
> --------------------------------------------------------------------------------
>
> Key: SPARK-13132
> URL: https://issues.apache.org/jira/browse/SPARK-13132
> Project: Spark
> Issue Type: Improvement
> Components: ML
> Affects Versions: 1.6.0
> Reporter: Gary King
> Assignee: Gary King
> Fix For: 2.0.0
>
>
> when L1 regularization is used, the inner functor passed to the quasi-newton optimizer in {{org.apache.spark.ml.classification.LogisticRegression#train}} makes repeated calls to {{$(standardization)}}. because this ultimately involves repeated string interpolation triggered by {{org.apache.spark.ml.param.Param#hashCode}}, this line of code consumes 35%-45% of the entire training time in my application.
> the range depends on whether the application sets an explicit value for the standardization parameter or relies on the default value (which needs an extra map lookup, resulting in an extra string interpolation, compared to the explicitly set case)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org