You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (JIRA)" <ji...@apache.org> on 2016/02/02 17:28:40 UTC
[jira] [Assigned] (SPARK-13132) LogisticRegression spends 35% of
its time fetching the standardization parameter
[ https://issues.apache.org/jira/browse/SPARK-13132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-13132:
------------------------------------
Assignee: Apache Spark
> LogisticRegression spends 35% of its time fetching the standardization parameter
> --------------------------------------------------------------------------------
>
> Key: SPARK-13132
> URL: https://issues.apache.org/jira/browse/SPARK-13132
> Project: Spark
> Issue Type: Improvement
> Components: ML
> Affects Versions: 1.6.0
> Reporter: Gary King
> Assignee: Apache Spark
>
> when L1 regularization is used, the inner functor passed to the quasi-newton optimizer in {{org.apache.spark.ml.classification.LogisticRegression#train}} makes repeated calls to {{$(standardization)}}. because this ultimately involves repeated string interpolation triggered by {{org.apache.spark.ml.param.Param#hashCode}}, this line of code consumes 35%-45% of the entire training time in my application.
> the range depends on whether the application sets an explicit value for the standardization parameter or relies on the default value (which needs an extra map lookup, resulting in an extra string interpolation, compared to the explicitly set case)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org