You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Weichen Xu (JIRA)" <ji...@apache.org> on 2016/07/20 04:38:20 UTC
[jira] [Closed] (SPARK-16638) The L2 regularization of
LinearRegression seems wrong when standardization is false
[ https://issues.apache.org/jira/browse/SPARK-16638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Weichen Xu closed SPARK-16638.
------------------------------
Resolution: Not A Problem
> The L2 regularization of LinearRegression seems wrong when standardization is false
> -----------------------------------------------------------------------------------
>
> Key: SPARK-16638
> URL: https://issues.apache.org/jira/browse/SPARK-16638
> Project: Spark
> Issue Type: Bug
> Components: ML, Optimizer
> Reporter: Weichen Xu
> Original Estimate: 1m
> Remaining Estimate: 1m
>
> The original L2 is
> 0.5 * effectiveL2regParam * sigma( wi^2 )
> (wi is the coefficients we want to train)
> And in linearRegression code, when standardization == false, the code modify L2 into:
> 0.5 * effectiveL2regParam * sigma( ( w[i] / featuresStd[i] )^2 )
> It is obviously wrong, I think.
> As the purpose of author wrote in the code comment, the modification to L2 reg should be:
> 0.5 * effectiveL2regParam * sigma( ( w[i] * featuresStd[i] )^2 )
> wi should not be divided by featuresStd[i], but should be multiplied by featuresStd[i]
> We can simply think this problem in the following way:
> assume a training data, there is a dimension k with very large std ( the value of featuresStd[k] is very large), so, we hope the training result coefficient w[k] to be small to keep the numerical stability. As a way of that, we can add a penalty to the L2 reg on this dimension.
> so that it should be surely w[k] * featuresStd[k], not w[k] / featuresStd[k]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org