You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by Lewuathe <gi...@git.apache.org> on 2015/09/23 13:59:06 UTC
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
GitHub user Lewuathe opened a pull request:
https://github.com/apache/spark/pull/8884
[SPARK-10668][ML] Use WeightedLeastSquares in LinearRegression with L…
…2 regularization if the number of features is small
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/Lewuathe/spark SPARK-10668
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/8884.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #8884
----
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-149126342
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by Lewuathe <gi...@git.apache.org>.
Github user Lewuathe commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-149385034
@dbtsai Thank you so much for long time catching up and reviewing again.
I'll create another follow-up JIRA of this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40519224
--- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -199,8 +238,8 @@ class LinearRegression(override val uid: String)
val costFun = new LeastSquaresCostFun(instances, yStd, yMean, $(fitIntercept),
$(standardization), featuresStd, featuresMean, effectiveL2RegParam)
-
- val optimizer = if ($(elasticNetParam) == 0.0 || effectiveRegParam == 0.0) {
--- End diff --
Why this has to be changed? Should we just keep it the same?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-149155399
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147281116
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41811124
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,559 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }}
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.24300
+ as.numeric.data.V2. 4.024821
+ as.numeric.data.V3. 6.679841
+ */
+ val interceptR1 = 6.24300
+ val weightsR1 = Vectors.dense(4.024821, 6.679841)
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.416948
+ as.numeric.data.V2. 3.893869
+ as.numeric.data.V3. 6.724286
+ */
+ val interceptR2 = 6.416948
+ val weightsR2 = Vectors.dense(3.893869, 6.724286)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.299752
- as.numeric.data.V3. 4.772913
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(6.299752, 4.772913)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.232193
- as.numeric.data.V3. 4.764229
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(6.232193, 4.764229)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.299752
+ as.numeric.data.V3. 4.772913
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(6.299752, 4.772913)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.232193
+ as.numeric.data.V3. 4.764229
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(6.232193, 4.764229)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression with intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.269376
- as.numeric.data.V2. 3.736216
- as.numeric.data.V3. 5.712356)
- */
- val interceptR1 = 5.269376
- val weightsR1 = Vectors.dense(3.736216, 5.712356)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.791109
- as.numeric.data.V2. 3.435466
- as.numeric.data.V3. 5.910406
- */
- val interceptR2 = 5.791109
- val weightsR2 = Vectors.dense(3.435466, 5.910406)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.269376
+ as.numeric.data.V2. 3.736216
+ as.numeric.data.V3. 5.712356)
+ */
+ val interceptR1 = 5.269376
+ val weightsR1 = Vectors.dense(3.736216, 5.712356)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.791109
+ as.numeric.data.V2. 3.435466
+ as.numeric.data.V3. 5.910406
+ */
+ val interceptR2 = 5.791109
+ val weightsR2 = Vectors.dense(3.435466, 5.910406)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.522875
- as.numeric.data.V3. 4.214502
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.522875, 4.214502)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.263704
- as.numeric.data.V3. 4.187419
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.263704, 4.187419)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.522875
+ as.numeric.data.V3. 4.214502
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.522875, 4.214502)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.263704
+ as.numeric.data.V3. 4.187419
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.263704, 4.187419)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression with intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.324108
- as.numeric.data.V2. 3.168435
- as.numeric.data.V3. 5.200403
- */
- val interceptR1 = 5.696056
- val weightsR1 = Vectors.dense(3.670489, 6.001122)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.114723
- as.numeric.data.V2. 3.409937
- as.numeric.data.V3. 6.146531
- */
- val interceptR2 = 6.114723
- val weightsR2 = Vectors.dense(3.409937, 6.146531)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.324108
+ as.numeric.data.V2. 3.168435
+ as.numeric.data.V3. 5.200403
+ */
+ val interceptR1 = 5.696056
+ val weightsR1 = Vectors.dense(3.670489, 6.001122)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.114723
+ as.numeric.data.V2. 3.409937
+ as.numeric.data.V3. 6.146531
+ */
+ val interceptR2 = 6.114723
+ val weightsR2 = Vectors.dense(3.409937, 6.146531)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
--- End diff --
ditto
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40518538
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -150,14 +155,14 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
as.numeric.data.V2. 6.995908
as.numeric.data.V3. 5.275131
*/
- val weightsR = Vectors.dense(6.995908, 5.275131)
+ val weightsR = Vectors.dense(6.995908, 5.275131)
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
- /*
+ /*
--- End diff --
ditto
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142586019
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42905/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-148679505
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-149127703
**[Test build #43920 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43920/consoleFull)** for PR 8884 at commit [`159f817`](https://github.com/apache/spark/commit/159f817b649fd794213fbad44abab962946c228e).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-149155084
**[Test build #43921 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43921/consoleFull)** for PR 8884 at commit [`c121e10`](https://github.com/apache/spark/commit/c121e10813b02be90c7ff695a98976a57589b7f6).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41808787
--- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -130,9 +131,54 @@ class LinearRegression(override val uid: String)
def setWeightCol(value: String): this.type = set(weightCol, value)
setDefault(weightCol -> "")
+ /**
+ * Set the solver algorithm used for optimization.
+ * In case of linear regression, this can be "l-bfgs", "normal" and "auto".
+ * The default value is "auto" which means that the solver algorithm is
+ * selected automatically.
+ * @group setParam
+ */
+ def setSolver(value: String): this.type = set(solver, value)
+ setDefault(solver -> "auto")
+
override protected def train(dataset: DataFrame): LinearRegressionModel = {
+ // Extract the number of features before deciding optimization solver.
+ val numFeatures = dataset.select(col($(featuresCol))).limit(1).map {
+ case Row(features: Vector) =>
+ features.size
+ }.toArray()(0)
// Extract columns from data. If dataset is persisted, do not persist instances.
val w = if ($(weightCol).isEmpty) lit(1.0) else col($(weightCol))
+
+ if ($(solver) == "normal" || ($(solver) == "auto"
+ && $(elasticNetParam) == 0.0 && numFeatures <= 4096)) {
--- End diff --
For readability,
```scala
if (($(solver) == "auto" && $(elasticNetParam) == 0.0 && numFeatures <= 4096) ||
$(solver) == "normal") {
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-148680732
[Test build #43839 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43839/consoleFull) for PR 8884 at commit [`c8d0ec9`](https://github.com/apache/spark/commit/c8d0ec9f2a96796c0ab1c9f7320985431ddcc1f1).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-143134212
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41810042
--- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -130,9 +131,54 @@ class LinearRegression(override val uid: String)
def setWeightCol(value: String): this.type = set(weightCol, value)
setDefault(weightCol -> "")
+ /**
+ * Set the solver algorithm used for optimization.
+ * In case of linear regression, this can be "l-bfgs", "normal" and "auto".
+ * The default value is "auto" which means that the solver algorithm is
+ * selected automatically.
+ * @group setParam
+ */
+ def setSolver(value: String): this.type = set(solver, value)
+ setDefault(solver -> "auto")
+
override protected def train(dataset: DataFrame): LinearRegressionModel = {
+ // Extract the number of features before deciding optimization solver.
+ val numFeatures = dataset.select(col($(featuresCol))).limit(1).map {
+ case Row(features: Vector) =>
+ features.size
+ }.toArray()(0)
// Extract columns from data. If dataset is persisted, do not persist instances.
val w = if ($(weightCol).isEmpty) lit(1.0) else col($(weightCol))
+
+ if ($(solver) == "normal" || ($(solver) == "auto"
+ && $(elasticNetParam) == 0.0 && numFeatures <= 4096)) {
+ require($(elasticNetParam) == 0.0, "Only L2 regularization can be used when normal " +
+ "solver is selected.'")
+ // In case of feature size is small, WeightedLeastSquares can train more efficiently
+ // because it requires one pass through to the data. (SPARK-10668)
+ val instances: RDD[WeightedLeastSquares.Instance] = dataset.select(
+ col($(labelCol)), w, col($(featuresCol))).map {
+ case Row(label: Double, weight: Double, features: Vector) =>
+ WeightedLeastSquares.Instance(weight, features, label)
+ }
+
+ val optimizer = new WeightedLeastSquares($(fitIntercept), $(regParam),
+ $(standardization), true)
+ val model = optimizer.fit(instances)
+ // When it is trained by WeightedLeastSquares, training summary does not
+ // attached returned model.
+ val lrModel = new LinearRegressionModel(uid, model.coefficients, model.intercept)
+ // WeightedLeastSquares does not run through iterations. So it does not generate
+ // an objective history.
+ val trainingSummary = new LinearRegressionTrainingSummary(
+ lrModel.transform(dataset),
+ $(predictionCol),
+ $(labelCol),
+ $(featuresCol),
+ Array(0D))
+ return copyValues(lrModel.setSummary(trainingSummary))
--- End diff --
Remove `copyValues`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-143134203
[Test build #43009 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43009/console) for PR 8884 at commit [`87626e4`](https://github.com/apache/spark/commit/87626e492ce5bca058efb4658d971987a515cecc).
* This patch **fails R style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r42401879
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,603 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with only L1 regularization case.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.24300
+ as.numeric.data.V2. 4.024821
+ as.numeric.data.V3. 6.679841
+ */
+ val interceptR1 = 6.24300
+ val weightsR1 = Vectors.dense(4.024821, 6.679841)
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.416948
+ as.numeric.data.V2. 3.893869
+ as.numeric.data.V3. 6.724286
+ */
+ val interceptR2 = 6.416948
+ val weightsR2 = Vectors.dense(3.893869, 6.724286)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression without intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.299752
- as.numeric.data.V3. 4.772913
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(6.299752, 4.772913)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.232193
- as.numeric.data.V3. 4.764229
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(6.232193, 4.764229)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
--- End diff --
ditto.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147911630
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41810413
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,559 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
--- End diff --
ditto
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41809540
--- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -130,9 +131,54 @@ class LinearRegression(override val uid: String)
def setWeightCol(value: String): this.type = set(weightCol, value)
setDefault(weightCol -> "")
+ /**
+ * Set the solver algorithm used for optimization.
+ * In case of linear regression, this can be "l-bfgs", "normal" and "auto".
+ * The default value is "auto" which means that the solver algorithm is
+ * selected automatically.
+ * @group setParam
+ */
+ def setSolver(value: String): this.type = set(solver, value)
+ setDefault(solver -> "auto")
+
override protected def train(dataset: DataFrame): LinearRegressionModel = {
+ // Extract the number of features before deciding optimization solver.
+ val numFeatures = dataset.select(col($(featuresCol))).limit(1).map {
+ case Row(features: Vector) =>
+ features.size
+ }.toArray()(0)
// Extract columns from data. If dataset is persisted, do not persist instances.
val w = if ($(weightCol).isEmpty) lit(1.0) else col($(weightCol))
+
+ if ($(solver) == "normal" || ($(solver) == "auto"
+ && $(elasticNetParam) == 0.0 && numFeatures <= 4096)) {
+ require($(elasticNetParam) == 0.0, "Only L2 regularization can be used when normal " +
+ "solver is selected.'")
+ // In case of feature size is small, WeightedLeastSquares can train more efficiently
+ // because it requires one pass through to the data. (SPARK-10668)
+ val instances: RDD[WeightedLeastSquares.Instance] = dataset.select(
+ col($(labelCol)), w, col($(featuresCol))).map {
+ case Row(label: Double, weight: Double, features: Vector) =>
+ WeightedLeastSquares.Instance(weight, features, label)
+ }
+
+ val optimizer = new WeightedLeastSquares($(fitIntercept), $(regParam),
+ $(standardization), true)
+ val model = optimizer.fit(instances)
+ // When it is trained by WeightedLeastSquares, training summary does not
--- End diff --
Is this comment still true?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by Lewuathe <gi...@git.apache.org>.
Github user Lewuathe commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r42339576
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +96,641 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with only L1 regularization case.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ break
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+ }
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.24300
+ as.numeric.data.V2. 4.024821
+ as.numeric.data.V3. 6.679841
+ */
+ val interceptR1 = 6.24300
+ val weightsR1 = Vectors.dense(4.024821, 6.679841)
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.416948
+ as.numeric.data.V2. 3.893869
+ as.numeric.data.V3. 6.724286
+ */
+ val interceptR2 = 6.416948
+ val weightsR2 = Vectors.dense(3.893869, 6.724286)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression without intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.299752
- as.numeric.data.V3. 4.772913
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(6.299752, 4.772913)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.232193
- as.numeric.data.V3. 4.764229
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(6.232193, 4.764229)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with only L1 regularization case.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ break
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+ }
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.299752
+ as.numeric.data.V3. 4.772913
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(6.299752, 4.772913)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.232193
+ as.numeric.data.V3. 4.764229
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(6.232193, 4.764229)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression with intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.269376
- as.numeric.data.V2. 3.736216
- as.numeric.data.V3. 5.712356)
- */
- val interceptR1 = 5.269376
- val weightsR1 = Vectors.dense(3.736216, 5.712356)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.791109
- as.numeric.data.V2. 3.435466
- as.numeric.data.V3. 5.910406
- */
- val interceptR2 = 5.791109
- val weightsR2 = Vectors.dense(3.435466, 5.910406)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.269376
+ as.numeric.data.V2. 3.736216
+ as.numeric.data.V3. 5.712356)
+ */
+ val interceptR1 = 5.269376
+ val weightsR1 = Vectors.dense(3.736216, 5.712356)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.791109
+ as.numeric.data.V2. 3.435466
+ as.numeric.data.V3. 5.910406
+ */
+ val interceptR2 = 5.791109
+ val weightsR2 = Vectors.dense(3.435466, 5.910406)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression without intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.522875
- as.numeric.data.V3. 4.214502
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.522875, 4.214502)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.263704
- as.numeric.data.V3. 4.187419
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.263704, 4.187419)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.522875
+ as.numeric.data.V3. 4.214502
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.522875, 4.214502)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.263704
+ as.numeric.data.V3. 4.187419
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.263704, 4.187419)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression with intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.324108
- as.numeric.data.V2. 3.168435
- as.numeric.data.V3. 5.200403
- */
- val interceptR1 = 5.696056
- val weightsR1 = Vectors.dense(3.670489, 6.001122)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.114723
- as.numeric.data.V2. 3.409937
- as.numeric.data.V3. 6.146531
- */
- val interceptR2 = 6.114723
- val weightsR2 = Vectors.dense(3.409937, 6.146531)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setStandardization(false).setSolver(solver)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with non-zero elasticnet parameter.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ break
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+ }
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.324108
+ as.numeric.data.V2. 3.168435
+ as.numeric.data.V3. 5.200403
+ */
+ val interceptR1 = 5.696056
+ val weightsR1 = Vectors.dense(3.670489, 6.001122)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.114723
+ as.numeric.data.V2. 3.409937
+ as.numeric.data.V3. 6.146531
+ */
+ val interceptR2 = 6.114723
+ val weightsR2 = Vectors.dense(3.409937, 6.146531)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression without intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.dataM.V2. 5.673348
- as.numeric.dataM.V3. 4.322251
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.673348, 4.322251)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.477988
- as.numeric.data.V3. 4.297622
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.477988, 4.297622)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with non-zero elasticnet parameter.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ break
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+ }
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.dataM.V2. 5.673348
+ as.numeric.dataM.V3. 4.322251
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.673348, 4.322251)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.477988
+ as.numeric.data.V3. 4.297622
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.477988, 4.297622)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression model training summary") {
- val trainer = new LinearRegression
- val model = trainer.fit(dataset)
- val trainerNoPredictionCol = trainer.setPredictionCol("")
- val modelNoPredictionCol = trainerNoPredictionCol.fit(dataset)
-
-
- // Training results for the model should be available
- assert(model.hasSummary)
- assert(modelNoPredictionCol.hasSummary)
-
- // Schema should be a superset of the input dataset
- assert((dataset.schema.fieldNames.toSet + "prediction").subsetOf(
- model.summary.predictions.schema.fieldNames.toSet))
- // Validate that we re-insert a prediction column for evaluation
- val modelNoPredictionColFieldNames = modelNoPredictionCol.summary.predictions.schema.fieldNames
- assert((dataset.schema.fieldNames.toSet).subsetOf(
- modelNoPredictionColFieldNames.toSet))
- assert(modelNoPredictionColFieldNames.exists(s => s.startsWith("prediction_")))
-
- // Residuals in [[LinearRegressionResults]] should equal those manually computed
- val expectedResiduals = dataset.select("features", "label")
- .map { case Row(features: DenseVector, label: Double) =>
- val prediction =
- features(0) * model.weights(0) + features(1) * model.weights(1) + model.intercept
- label - prediction
- }
- .zip(model.summary.residuals.map(_.getDouble(0)))
- .collect()
- .foreach { case (manualResidual: Double, resultResidual: Double) =>
- assert(manualResidual ~== resultResidual relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer = new LinearRegression().setSolver(solver)
+ val model = trainer.fit(dataset)
+ val trainerNoPredictionCol = trainer.setPredictionCol("")
+ val modelNoPredictionCol = trainerNoPredictionCol.fit(dataset)
+
+
+ // Training results for the model should be available
+ assert(model.hasSummary)
+ assert(modelNoPredictionCol.hasSummary)
+
+ // Schema should be a superset of the input dataset
+ assert((dataset.schema.fieldNames.toSet + "prediction").subsetOf(
+ model.summary.predictions.schema.fieldNames.toSet))
+ // Validate that we re-insert a prediction column for evaluation
+ val modelNoPredictionColFieldNames
+ = modelNoPredictionCol.summary.predictions.schema.fieldNames
+ assert((dataset.schema.fieldNames.toSet).subsetOf(
+ modelNoPredictionColFieldNames.toSet))
+ assert(modelNoPredictionColFieldNames.exists(s => s.startsWith("prediction_")))
+
+ // Residuals in [[LinearRegressionResults]] should equal those manually computed
+ val expectedResiduals = dataset.select("features", "label")
+ .map { case Row(features: DenseVector, label: Double) =>
+ val prediction =
+ features(0) * model.weights(0) + features(1) * model.weights(1) + model.intercept
+ label - prediction
+ }
+ .zip(model.summary.residuals.map(_.getDouble(0)))
+ .collect()
+ .foreach { case (manualResidual: Double, resultResidual: Double) =>
+ assert(manualResidual ~== resultResidual relTol 1E-5)
+ }
- /*
- Use the following R code to generate model training results.
-
- predictions <- predict(fit, newx=features)
- residuals <- label - predictions
- > mean(residuals^2) # MSE
- [1] 0.009720325
- > mean(abs(residuals)) # MAD
- [1] 0.07863206
- > cor(predictions, label)^2# r^2
- [,1]
- s0 0.9998749
- */
- assert(model.summary.meanSquaredError ~== 0.00972035 relTol 1E-5)
- assert(model.summary.meanAbsoluteError ~== 0.07863206 relTol 1E-5)
- assert(model.summary.r2 ~== 0.9998749 relTol 1E-5)
-
- // Objective function should be monotonically decreasing for linear regression
- assert(
- model.summary
- .objectiveHistory
- .sliding(2)
- .forall(x => x(0) >= x(1)))
+ /*
+ Use the following R code to generate model training results.
+
+ predictions <- predict(fit, newx=features)
+ residuals <- label - predictions
+ > mean(residuals^2) # MSE
+ [1] 0.009720325
+ > mean(abs(residuals)) # MAD
+ [1] 0.07863206
+ > cor(predictions, label)^2# r^2
+ [,1]
+ s0 0.9998749
+ */
+ assert(model.summary.meanSquaredError ~== 0.00972035 relTol 1E-5)
+ assert(model.summary.meanAbsoluteError ~== 0.07863206 relTol 1E-5)
+ assert(model.summary.r2 ~== 0.9998749 relTol 1E-5)
+
+ // Normal solver uses "WeightedLeastSquares". This algorithm does not generate
+ // objective history because it does not run through iterations.
+ if (solver == "l-bfgs") {
+ // Objective function should be monotonically decreasing for linear regression
+ assert(
+ model.summary
+ .objectiveHistory
+ .sliding(2)
+ .forall(x => x(0) >= x(1)))
+ }
+ }
}
test("linear regression model testset evaluation summary") {
- val trainer = new LinearRegression
- val model = trainer.fit(dataset)
-
- // Evaluating on training dataset should yield results summary equal to training summary
- val testSummary = model.evaluate(dataset)
- assert(model.summary.meanSquaredError ~== testSummary.meanSquaredError relTol 1E-5)
- assert(model.summary.r2 ~== testSummary.r2 relTol 1E-5)
- model.summary.residuals.select("residuals").collect()
- .zip(testSummary.residuals.select("residuals").collect())
- .forall { case (Row(r1: Double), Row(r2: Double)) => r1 ~== r2 relTol 1E-5 }
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer = new LinearRegression().setSolver(solver)
+ val model = trainer.fit(dataset)
+
+ // Evaluating on training dataset should yield results summary equal to training summary
+ val testSummary = model.evaluate(dataset)
+ assert(model.summary.meanSquaredError ~== testSummary.meanSquaredError relTol 1E-5)
+ assert(model.summary.r2 ~== testSummary.r2 relTol 1E-5)
+ model.summary.residuals.select("residuals").collect()
+ .zip(testSummary.residuals.select("residuals").collect())
+ .forall { case (Row(r1: Double), Row(r2: Double)) => r1 ~== r2 relTol 1E-5 }
+ }
}
test("linear regression with weighted samples"){
- val (data, weightedData) = {
- val activeData = LinearDataGenerator.generateLinearInput(
- 6.3, Array(4.7, 7.2), Array(0.9, -1.3), Array(0.7, 1.2), 500, 1, 0.1)
-
- val rnd = new Random(8392)
- val signedData = activeData.map { case p: LabeledPoint =>
- (rnd.nextGaussian() > 0.0, p)
- }
-
- val data1 = signedData.flatMap {
- case (true, p) => Iterator(p, p)
- case (false, p) => Iterator(p)
- }
-
- val weightedSignedData = signedData.flatMap {
- case (true, LabeledPoint(label, features)) =>
- Iterator(
- Instance(label, weight = 1.2, features),
- Instance(label, weight = 0.8, features)
- )
- case (false, LabeledPoint(label, features)) =>
- Iterator(
- Instance(label, weight = 0.3, features),
- Instance(label, weight = 0.1, features),
- Instance(label, weight = 0.6, features)
- )
- }
-
- val noiseData = LinearDataGenerator.generateLinearInput(
- 2, Array(1, 3), Array(0.9, -1.3), Array(0.7, 1.2), 500, 1, 0.1)
- val weightedNoiseData = noiseData.map {
- case LabeledPoint(label, features) => Instance(label, weight = 0, features)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val (data, weightedData) = {
+ val activeData = LinearDataGenerator.generateLinearInput(
+ 6.3, Array(4.7, 7.2), Array(0.9, -1.3), Array(0.7, 1.2), 500, 1, 0.1)
+
+ val rnd = new Random(8392)
+ val signedData = activeData.map { case p: LabeledPoint =>
+ (rnd.nextGaussian() > 0.0, p)
+ }
+
+ val data1 = signedData.flatMap {
+ case (true, p) => Iterator(p, p)
+ case (false, p) => Iterator(p)
+ }
+
+ val weightedSignedData = signedData.flatMap {
+ case (true, LabeledPoint(label, features)) =>
+ Iterator(
+ Instance(label, weight = 1.2, features),
+ Instance(label, weight = 0.8, features)
+ )
+ case (false, LabeledPoint(label, features)) =>
+ Iterator(
+ Instance(label, weight = 0.3, features),
+ Instance(label, weight = 0.1, features),
+ Instance(label, weight = 0.6, features)
+ )
+ }
+
+ val noiseData = LinearDataGenerator.generateLinearInput(
+ 2, Array(1, 3), Array(0.9, -1.3), Array(0.7, 1.2), 500, 1, 0.1)
+ val weightedNoiseData = noiseData.map {
+ case LabeledPoint(label, features) => Instance(label, weight = 0, features)
+ }
+ val data2 = weightedSignedData ++ weightedNoiseData
+
+ (sqlContext.createDataFrame(sc.parallelize(data1, 4)),
+ sqlContext.createDataFrame(sc.parallelize(data2, 4)))
+ }
+
+ val trainer1a = (new LinearRegression).setFitIntercept(true)
+ .setElasticNetParam(0.38).setRegParam(0.21).setStandardization(true).setSolver(solver)
+ val trainer1b = (new LinearRegression).setFitIntercept(true).setWeightCol("weight")
+ .setElasticNetParam(0.38).setRegParam(0.21).setStandardization(true).setSolver(solver)
+
+ var model1a0: LinearRegressionModel = null
+ var model1a1: LinearRegressionModel = null
+ var model1b: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with non-zero elasticnet parameter.
+ if (solver == "normal") {
--- End diff --
Okay, thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41810438
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,559 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }}
--- End diff --
ditto
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147712183
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-149131660
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-143185352
[Test build #43021 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43021/console) for PR 8884 at commit [`98be936`](https://github.com/apache/spark/commit/98be9360c5daf9df8b841d290f1f3e899a9a7617).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-149007582
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43897/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41808876
--- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -130,9 +131,54 @@ class LinearRegression(override val uid: String)
def setWeightCol(value: String): this.type = set(weightCol, value)
setDefault(weightCol -> "")
+ /**
+ * Set the solver algorithm used for optimization.
+ * In case of linear regression, this can be "l-bfgs", "normal" and "auto".
+ * The default value is "auto" which means that the solver algorithm is
+ * selected automatically.
+ * @group setParam
+ */
+ def setSolver(value: String): this.type = set(solver, value)
+ setDefault(solver -> "auto")
+
override protected def train(dataset: DataFrame): LinearRegressionModel = {
+ // Extract the number of features before deciding optimization solver.
+ val numFeatures = dataset.select(col($(featuresCol))).limit(1).map {
+ case Row(features: Vector) =>
+ features.size
+ }.toArray()(0)
// Extract columns from data. If dataset is persisted, do not persist instances.
val w = if ($(weightCol).isEmpty) lit(1.0) else col($(weightCol))
+
+ if ($(solver) == "normal" || ($(solver) == "auto"
+ && $(elasticNetParam) == 0.0 && numFeatures <= 4096)) {
+ require($(elasticNetParam) == 0.0, "Only L2 regularization can be used when normal " +
+ "solver is selected.'")
--- End diff --
Since in `auto` mode, users don't select solver explicitly. Change `selected` to `used`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-149131593
**[Test build #43920 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43920/consoleFull)** for PR 8884 at commit [`159f817`](https://github.com/apache/spark/commit/159f817b649fd794213fbad44abab962946c228e).
* This patch **fails MiMa tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147928871
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43700/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by yanboliang <gi...@git.apache.org>.
Github user yanboliang commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40529250
--- Diff: mllib/src/main/scala/org/apache/spark/ml/param/shared/sharedParams.scala ---
@@ -357,4 +357,17 @@ private[ml] trait HasWeightCol extends Params {
/** @group getParam */
final def getWeightCol: String = $(weightCol)
}
+
+/**
+ * Trait for the solver algorithm for optimization.
+ */
+private[ml] trait HasSolver extends Params {
+ /**
+ * Param for solver algorithm. If this is not set or empty, we select the solver algorithm automatically.
+ */
+ final val solver: Param[String] = new Param[String](this, "solver", "the solver algorithm for optimization. If this is not set or empty, default value is 'auto'.")
--- End diff --
Shall we have a validation function here to check the ```solver``` is set with legal value? Such as if users set ```solver``` with ```normal``` but the number of features is more than 4096, throwing exception here. [WeightedLeastSquares](https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/ml/optim/WeightedLeastSquares.scala#L173) will check this after job starting, but I think we should detect this error earlier.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by Lewuathe <gi...@git.apache.org>.
Github user Lewuathe commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-148227686
@dbtsai Thanks again. Could check it?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142750616
[Test build #42929 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42929/consoleFull) for PR 8884 at commit [`5ca2a39`](https://github.com/apache/spark/commit/5ca2a3980a5a3820b125ab84481382edcf386c73).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142758267
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142604970
[Test build #42907 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42907/console) for PR 8884 at commit [`5ca2a39`](https://github.com/apache/spark/commit/5ca2a3980a5a3820b125ab84481382edcf386c73).
* This patch **fails MiMa tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40519092
--- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -169,6 +181,33 @@ class LinearRegression(override val uid: String)
val yMean = ySummarizer.mean(0)
val yStd = math.sqrt(ySummarizer.variance(0))
--- End diff --
1) `instances` will be cached, but if we go the `normal` path, `instances` will be never unpersisted.
2) Please move the if condition to the top, and if the solver is `auto`, we just do a take(1) to get number of features so we don't run unnecessary `val (featuresSummarizer, ySummarizer) `. This can solve 1) as well.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142585980
[Test build #42905 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42905/console) for PR 8884 at commit [`570a558`](https://github.com/apache/spark/commit/570a5588858d690c240d5baa979fa2510f74b705).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-144386013
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r42402047
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,603 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with only L1 regularization case.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.24300
+ as.numeric.data.V2. 4.024821
+ as.numeric.data.V3. 6.679841
+ */
+ val interceptR1 = 6.24300
+ val weightsR1 = Vectors.dense(4.024821, 6.679841)
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.416948
+ as.numeric.data.V2. 3.893869
+ as.numeric.data.V3. 6.724286
+ */
+ val interceptR2 = 6.416948
+ val weightsR2 = Vectors.dense(3.893869, 6.724286)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression without intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.299752
- as.numeric.data.V3. 4.772913
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(6.299752, 4.772913)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.232193
- as.numeric.data.V3. 4.764229
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(6.232193, 4.764229)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with only L1 regularization case.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.299752
+ as.numeric.data.V3. 4.772913
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(6.299752, 4.772913)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.232193
+ as.numeric.data.V3. 4.764229
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(6.232193, 4.764229)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression with intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.269376
- as.numeric.data.V2. 3.736216
- as.numeric.data.V3. 5.712356)
- */
- val interceptR1 = 5.269376
- val weightsR1 = Vectors.dense(3.736216, 5.712356)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.791109
- as.numeric.data.V2. 3.435466
- as.numeric.data.V3. 5.910406
- */
- val interceptR2 = 5.791109
- val weightsR2 = Vectors.dense(3.435466, 5.910406)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.269376
+ as.numeric.data.V2. 3.736216
+ as.numeric.data.V3. 5.712356)
+ */
+ val interceptR1 = 5.269376
+ val weightsR1 = Vectors.dense(3.736216, 5.712356)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.791109
+ as.numeric.data.V2. 3.435466
+ as.numeric.data.V3. 5.910406
+ */
+ val interceptR2 = 5.791109
+ val weightsR2 = Vectors.dense(3.435466, 5.910406)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression without intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.522875
- as.numeric.data.V3. 4.214502
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.522875, 4.214502)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.263704
- as.numeric.data.V3. 4.187419
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.263704, 4.187419)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.522875
+ as.numeric.data.V3. 4.214502
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.522875, 4.214502)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.263704
+ as.numeric.data.V3. 4.187419
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.263704, 4.187419)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression with intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.324108
- as.numeric.data.V2. 3.168435
- as.numeric.data.V3. 5.200403
- */
- val interceptR1 = 5.696056
- val weightsR1 = Vectors.dense(3.670489, 6.001122)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.114723
- as.numeric.data.V2. 3.409937
- as.numeric.data.V3. 6.146531
- */
- val interceptR2 = 6.114723
- val weightsR2 = Vectors.dense(3.409937, 6.146531)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setStandardization(false).setSolver(solver)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with non-zero elasticnet parameter.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.324108
+ as.numeric.data.V2. 3.168435
+ as.numeric.data.V3. 5.200403
+ */
+ val interceptR1 = 5.696056
+ val weightsR1 = Vectors.dense(3.670489, 6.001122)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.114723
+ as.numeric.data.V2. 3.409937
+ as.numeric.data.V3. 6.146531
+ */
+ val interceptR2 = 6.114723
+ val weightsR2 = Vectors.dense(3.409937, 6.146531)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression without intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.dataM.V2. 5.673348
- as.numeric.dataM.V3. 4.322251
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.673348, 4.322251)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.477988
- as.numeric.data.V3. 4.297622
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.477988, 4.297622)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with non-zero elasticnet parameter.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.dataM.V2. 5.673348
+ as.numeric.dataM.V3. 4.322251
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.673348, 4.322251)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.477988
+ as.numeric.data.V3. 4.297622
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.477988, 4.297622)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression model training summary") {
- val trainer = new LinearRegression
- val model = trainer.fit(dataset)
- val trainerNoPredictionCol = trainer.setPredictionCol("")
- val modelNoPredictionCol = trainerNoPredictionCol.fit(dataset)
-
-
- // Training results for the model should be available
- assert(model.hasSummary)
- assert(modelNoPredictionCol.hasSummary)
-
- // Schema should be a superset of the input dataset
- assert((dataset.schema.fieldNames.toSet + "prediction").subsetOf(
- model.summary.predictions.schema.fieldNames.toSet))
- // Validate that we re-insert a prediction column for evaluation
- val modelNoPredictionColFieldNames = modelNoPredictionCol.summary.predictions.schema.fieldNames
- assert((dataset.schema.fieldNames.toSet).subsetOf(
- modelNoPredictionColFieldNames.toSet))
- assert(modelNoPredictionColFieldNames.exists(s => s.startsWith("prediction_")))
-
- // Residuals in [[LinearRegressionResults]] should equal those manually computed
- val expectedResiduals = dataset.select("features", "label")
- .map { case Row(features: DenseVector, label: Double) =>
- val prediction =
- features(0) * model.weights(0) + features(1) * model.weights(1) + model.intercept
- label - prediction
- }
- .zip(model.summary.residuals.map(_.getDouble(0)))
- .collect()
- .foreach { case (manualResidual: Double, resultResidual: Double) =>
- assert(manualResidual ~== resultResidual relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer = new LinearRegression().setSolver(solver)
+ val model = trainer.fit(dataset)
+ val trainerNoPredictionCol = trainer.setPredictionCol("")
+ val modelNoPredictionCol = trainerNoPredictionCol.fit(dataset)
+
+
--- End diff --
extra line
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-143085746
Vote for adding a new param called "solver". In the unit test, we have to test two code paths as well; otherwise, the previous LBFGS behavior will not be tested after this PR.
Please run the summary even use `WeightedLeastSquare`. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40518507
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -113,34 +115,37 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
as.numeric.data.V2. 4.700706
as.numeric.data.V3. 7.199082
*/
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ })
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
+ Seq("auto", "l-bfgs", "normal").foreach(solver => {
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
--- End diff --
remove extra line
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-149126356
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142790875
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41811201
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,559 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }}
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.24300
+ as.numeric.data.V2. 4.024821
+ as.numeric.data.V3. 6.679841
+ */
+ val interceptR1 = 6.24300
+ val weightsR1 = Vectors.dense(4.024821, 6.679841)
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.416948
+ as.numeric.data.V2. 3.893869
+ as.numeric.data.V3. 6.724286
+ */
+ val interceptR2 = 6.416948
+ val weightsR2 = Vectors.dense(3.893869, 6.724286)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.299752
- as.numeric.data.V3. 4.772913
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(6.299752, 4.772913)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.232193
- as.numeric.data.V3. 4.764229
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(6.232193, 4.764229)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.299752
+ as.numeric.data.V3. 4.772913
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(6.299752, 4.772913)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.232193
+ as.numeric.data.V3. 4.764229
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(6.232193, 4.764229)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression with intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.269376
- as.numeric.data.V2. 3.736216
- as.numeric.data.V3. 5.712356)
- */
- val interceptR1 = 5.269376
- val weightsR1 = Vectors.dense(3.736216, 5.712356)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.791109
- as.numeric.data.V2. 3.435466
- as.numeric.data.V3. 5.910406
- */
- val interceptR2 = 5.791109
- val weightsR2 = Vectors.dense(3.435466, 5.910406)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.269376
+ as.numeric.data.V2. 3.736216
+ as.numeric.data.V3. 5.712356)
+ */
+ val interceptR1 = 5.269376
+ val weightsR1 = Vectors.dense(3.736216, 5.712356)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.791109
+ as.numeric.data.V2. 3.435466
+ as.numeric.data.V3. 5.910406
+ */
+ val interceptR2 = 5.791109
+ val weightsR2 = Vectors.dense(3.435466, 5.910406)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.522875
- as.numeric.data.V3. 4.214502
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.522875, 4.214502)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.263704
- as.numeric.data.V3. 4.187419
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.263704, 4.187419)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.522875
+ as.numeric.data.V3. 4.214502
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.522875, 4.214502)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.263704
+ as.numeric.data.V3. 4.187419
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.263704, 4.187419)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression with intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.324108
- as.numeric.data.V2. 3.168435
- as.numeric.data.V3. 5.200403
- */
- val interceptR1 = 5.696056
- val weightsR1 = Vectors.dense(3.670489, 6.001122)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.114723
- as.numeric.data.V2. 3.409937
- as.numeric.data.V3. 6.146531
- */
- val interceptR2 = 6.114723
- val weightsR2 = Vectors.dense(3.409937, 6.146531)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.324108
+ as.numeric.data.V2. 3.168435
+ as.numeric.data.V3. 5.200403
+ */
+ val interceptR1 = 5.696056
+ val weightsR1 = Vectors.dense(3.670489, 6.001122)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.114723
+ as.numeric.data.V2. 3.409937
+ as.numeric.data.V3. 6.146531
+ */
+ val interceptR2 = 6.114723
+ val weightsR2 = Vectors.dense(3.409937, 6.146531)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.dataM.V2. 5.673348
- as.numeric.dataM.V3. 4.322251
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.673348, 4.322251)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.477988
- as.numeric.data.V3. 4.297622
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.477988, 4.297622)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.dataM.V2. 5.673348
+ as.numeric.dataM.V3. 4.322251
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.673348, 4.322251)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.477988
+ as.numeric.data.V3. 4.297622
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.477988, 4.297622)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression model training summary") {
- val trainer = new LinearRegression
- val model = trainer.fit(dataset)
- val trainerNoPredictionCol = trainer.setPredictionCol("")
- val modelNoPredictionCol = trainerNoPredictionCol.fit(dataset)
-
-
- // Training results for the model should be available
- assert(model.hasSummary)
- assert(modelNoPredictionCol.hasSummary)
-
- // Schema should be a superset of the input dataset
- assert((dataset.schema.fieldNames.toSet + "prediction").subsetOf(
- model.summary.predictions.schema.fieldNames.toSet))
- // Validate that we re-insert a prediction column for evaluation
- val modelNoPredictionColFieldNames = modelNoPredictionCol.summary.predictions.schema.fieldNames
- assert((dataset.schema.fieldNames.toSet).subsetOf(
- modelNoPredictionColFieldNames.toSet))
- assert(modelNoPredictionColFieldNames.exists(s => s.startsWith("prediction_")))
-
- // Residuals in [[LinearRegressionResults]] should equal those manually computed
- val expectedResiduals = dataset.select("features", "label")
- .map { case Row(features: DenseVector, label: Double) =>
- val prediction =
- features(0) * model.weights(0) + features(1) * model.weights(1) + model.intercept
- label - prediction
- }
- .zip(model.summary.residuals.map(_.getDouble(0)))
- .collect()
- .foreach { case (manualResidual: Double, resultResidual: Double) =>
- assert(manualResidual ~== resultResidual relTol 1E-5)
- }
+ Seq("l-bfgs").foreach { solver => {
--- End diff --
Now, all of them should have summary. Also remove the last `{`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41811114
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,559 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }}
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.24300
+ as.numeric.data.V2. 4.024821
+ as.numeric.data.V3. 6.679841
+ */
+ val interceptR1 = 6.24300
+ val weightsR1 = Vectors.dense(4.024821, 6.679841)
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.416948
+ as.numeric.data.V2. 3.893869
+ as.numeric.data.V3. 6.724286
+ */
+ val interceptR2 = 6.416948
+ val weightsR2 = Vectors.dense(3.893869, 6.724286)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.299752
- as.numeric.data.V3. 4.772913
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(6.299752, 4.772913)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.232193
- as.numeric.data.V3. 4.764229
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(6.232193, 4.764229)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.299752
+ as.numeric.data.V3. 4.772913
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(6.299752, 4.772913)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.232193
+ as.numeric.data.V3. 4.764229
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(6.232193, 4.764229)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression with intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.269376
- as.numeric.data.V2. 3.736216
- as.numeric.data.V3. 5.712356)
- */
- val interceptR1 = 5.269376
- val weightsR1 = Vectors.dense(3.736216, 5.712356)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.791109
- as.numeric.data.V2. 3.435466
- as.numeric.data.V3. 5.910406
- */
- val interceptR2 = 5.791109
- val weightsR2 = Vectors.dense(3.435466, 5.910406)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.269376
+ as.numeric.data.V2. 3.736216
+ as.numeric.data.V3. 5.712356)
+ */
+ val interceptR1 = 5.269376
+ val weightsR1 = Vectors.dense(3.736216, 5.712356)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.791109
+ as.numeric.data.V2. 3.435466
+ as.numeric.data.V3. 5.910406
+ */
+ val interceptR2 = 5.791109
+ val weightsR2 = Vectors.dense(3.435466, 5.910406)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.522875
- as.numeric.data.V3. 4.214502
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.522875, 4.214502)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.263704
- as.numeric.data.V3. 4.187419
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.263704, 4.187419)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.522875
+ as.numeric.data.V3. 4.214502
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.522875, 4.214502)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.263704
+ as.numeric.data.V3. 4.187419
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.263704, 4.187419)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression with intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.324108
- as.numeric.data.V2. 3.168435
- as.numeric.data.V3. 5.200403
- */
- val interceptR1 = 5.696056
- val weightsR1 = Vectors.dense(3.670489, 6.001122)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.114723
- as.numeric.data.V2. 3.409937
- as.numeric.data.V3. 6.146531
- */
- val interceptR2 = 6.114723
- val weightsR2 = Vectors.dense(3.409937, 6.146531)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
--- End diff --
ditto.
`Seq("auto", "l-bfgs", "normal").foreach { solver =>`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by Lewuathe <gi...@git.apache.org>.
Github user Lewuathe commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-148657383
@dbtsai Thank you for polite reviewing! I'll update.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-143133085
[Test build #43009 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43009/consoleFull) for PR 8884 at commit [`87626e4`](https://github.com/apache/spark/commit/87626e492ce5bca058efb4658d971987a515cecc).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40518652
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -206,30 +213,32 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
as.numeric.data.V2. 3.893869
as.numeric.data.V3. 6.724286
*/
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
+ val interceptR2 = 6.416948
+ val weightsR2 = Vectors.dense(3.893869, 6.724286)
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ })
}
test("linear regression without intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
+ Seq("auto").foreach(solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
--- End diff --
ditto. `foreach { ... `
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40518708
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -396,29 +413,31 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
as.numeric.data.V2. 3.409937
as.numeric.data.V3. 6.146531
*/
- val interceptR2 = 6.114723
- val weightsR2 = Vectors.dense(3.409937, 6.146531)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ val interceptR2 = 6.114723
+ val weightsR2 = Vectors.dense(3.409937, 6.146531)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ })
}
test("linear regression without intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
+ Seq("auto").foreach(solver => {
--- End diff --
ditto
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41811137
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,559 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }}
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.24300
+ as.numeric.data.V2. 4.024821
+ as.numeric.data.V3. 6.679841
+ */
+ val interceptR1 = 6.24300
+ val weightsR1 = Vectors.dense(4.024821, 6.679841)
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.416948
+ as.numeric.data.V2. 3.893869
+ as.numeric.data.V3. 6.724286
+ */
+ val interceptR2 = 6.416948
+ val weightsR2 = Vectors.dense(3.893869, 6.724286)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.299752
- as.numeric.data.V3. 4.772913
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(6.299752, 4.772913)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.232193
- as.numeric.data.V3. 4.764229
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(6.232193, 4.764229)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.299752
+ as.numeric.data.V3. 4.772913
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(6.299752, 4.772913)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.232193
+ as.numeric.data.V3. 4.764229
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(6.232193, 4.764229)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression with intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.269376
- as.numeric.data.V2. 3.736216
- as.numeric.data.V3. 5.712356)
- */
- val interceptR1 = 5.269376
- val weightsR1 = Vectors.dense(3.736216, 5.712356)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.791109
- as.numeric.data.V2. 3.435466
- as.numeric.data.V3. 5.910406
- */
- val interceptR2 = 5.791109
- val weightsR2 = Vectors.dense(3.435466, 5.910406)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.269376
+ as.numeric.data.V2. 3.736216
+ as.numeric.data.V3. 5.712356)
+ */
+ val interceptR1 = 5.269376
+ val weightsR1 = Vectors.dense(3.736216, 5.712356)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.791109
+ as.numeric.data.V2. 3.435466
+ as.numeric.data.V3. 5.910406
+ */
+ val interceptR2 = 5.791109
+ val weightsR2 = Vectors.dense(3.435466, 5.910406)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.522875
- as.numeric.data.V3. 4.214502
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.522875, 4.214502)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.263704
- as.numeric.data.V3. 4.187419
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.263704, 4.187419)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.522875
+ as.numeric.data.V3. 4.214502
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.522875, 4.214502)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.263704
+ as.numeric.data.V3. 4.187419
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.263704, 4.187419)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression with intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.324108
- as.numeric.data.V2. 3.168435
- as.numeric.data.V3. 5.200403
- */
- val interceptR1 = 5.696056
- val weightsR1 = Vectors.dense(3.670489, 6.001122)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.114723
- as.numeric.data.V2. 3.409937
- as.numeric.data.V3. 6.146531
- */
- val interceptR2 = 6.114723
- val weightsR2 = Vectors.dense(3.409937, 6.146531)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.324108
+ as.numeric.data.V2. 3.168435
+ as.numeric.data.V3. 5.200403
+ */
+ val interceptR1 = 5.696056
+ val weightsR1 = Vectors.dense(3.670489, 6.001122)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.114723
+ as.numeric.data.V2. 3.409937
+ as.numeric.data.V3. 6.146531
+ */
+ val interceptR2 = 6.114723
+ val weightsR2 = Vectors.dense(3.409937, 6.146531)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.dataM.V2. 5.673348
- as.numeric.dataM.V3. 4.322251
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.673348, 4.322251)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.477988
- as.numeric.data.V3. 4.297622
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.477988, 4.297622)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
--- End diff --
ditto
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-143179460
[Test build #43021 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43021/consoleFull) for PR 8884 at commit [`98be936`](https://github.com/apache/spark/commit/98be9360c5daf9df8b841d290f1f3e899a9a7617).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40518874
--- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -140,6 +141,17 @@ class LinearRegression(override val uid: String)
def setWeightCol(value: String): this.type = set(weightCol, value)
setDefault(weightCol -> "")
+ /**
+ * Set the solver algorithm for optimization.
+ * In case of linear regression, this can be "l-bfgs", "normal" and "auto".
+ * The default value is "auto" which means that the solver algorithm is
+ * selected automatically.
+ * @param value
--- End diff --
This should be
` * @group setParam`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147286851
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-143178133
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-148691353
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147726704
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40518823
--- Diff: mllib/src/main/scala/org/apache/spark/ml/param/shared/sharedParams.scala ---
@@ -357,4 +357,17 @@ private[ml] trait HasWeightCol extends Params {
/** @group getParam */
--- End diff --
This file can not be modified directly. Please add `HasSolver` param to `SharedParamsCodeGen.scala`, and then run `build/sbt "mllib/runMain org.apache.spark.ml.param.shared.SharedParamsCodeGen"` to generate `sharedParams.scala`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-144386015
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43130/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147186459
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43542/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by Lewuathe <gi...@git.apache.org>.
Github user Lewuathe commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40530056
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -150,14 +155,14 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
as.numeric.data.V2. 6.995908
as.numeric.data.V3. 5.275131
*/
- val weightsR = Vectors.dense(6.995908, 5.275131)
+ val weightsR = Vectors.dense(6.995908, 5.275131)
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
+ assert(model1.intercept ~== 0 absTol 1E-3)
--- End diff --
These lines become inside of `foreach` block. So these are not indentation issues.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142597921
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147910843
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-149295691
Thanks. Merged into master.
Please submit another PR to address the minor issues in the comment. We would like to see that when solver is `auto`, the `normal` solver will be picked up. This can be done by checking if the loss history is zero or not. Also, add another test case showing that when `n > 4096`, `l-bfgs will be picked up.
Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by Lewuathe <gi...@git.apache.org>.
Github user Lewuathe commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147323507
@dbtsai I rebased master and updated. Could you check it again? Thank you.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142783205
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147184278
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147712151
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-143120493
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41811067
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,559 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }}
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.24300
+ as.numeric.data.V2. 4.024821
+ as.numeric.data.V3. 6.679841
+ */
+ val interceptR1 = 6.24300
+ val weightsR1 = Vectors.dense(4.024821, 6.679841)
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.416948
+ as.numeric.data.V2. 3.893869
+ as.numeric.data.V3. 6.724286
+ */
+ val interceptR2 = 6.416948
+ val weightsR2 = Vectors.dense(3.893869, 6.724286)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.299752
- as.numeric.data.V3. 4.772913
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(6.299752, 4.772913)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.232193
- as.numeric.data.V3. 4.764229
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(6.232193, 4.764229)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
--- End diff --
ditto
`Seq("auto", "l-bfgs", "normal").foreach { solver =>`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147186391
[Test build #43542 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43542/console) for PR 8884 at commit [`7317aa2`](https://github.com/apache/spark/commit/7317aa255960a258e6b480188fe80a1e9454f461).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by Lewuathe <gi...@git.apache.org>.
Github user Lewuathe commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41860463
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,559 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }}
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
--- End diff --
`intercept` does not exit itself when it succeeded. So I think it might be a little hacky to write break code inside `intercept`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147915090
[Test build #43700 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43700/consoleFull) for PR 8884 at commit [`23a8ab1`](https://github.com/apache/spark/commit/23a8ab148f196ca2716eca890d68e5d3d21c85c3).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41811297
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,559 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }}
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.24300
+ as.numeric.data.V2. 4.024821
+ as.numeric.data.V3. 6.679841
+ */
+ val interceptR1 = 6.24300
+ val weightsR1 = Vectors.dense(4.024821, 6.679841)
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.416948
+ as.numeric.data.V2. 3.893869
+ as.numeric.data.V3. 6.724286
+ */
+ val interceptR2 = 6.416948
+ val weightsR2 = Vectors.dense(3.893869, 6.724286)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.299752
- as.numeric.data.V3. 4.772913
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(6.299752, 4.772913)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.232193
- as.numeric.data.V3. 4.764229
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(6.232193, 4.764229)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.299752
+ as.numeric.data.V3. 4.772913
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(6.299752, 4.772913)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.232193
+ as.numeric.data.V3. 4.764229
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(6.232193, 4.764229)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression with intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.269376
- as.numeric.data.V2. 3.736216
- as.numeric.data.V3. 5.712356)
- */
- val interceptR1 = 5.269376
- val weightsR1 = Vectors.dense(3.736216, 5.712356)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.791109
- as.numeric.data.V2. 3.435466
- as.numeric.data.V3. 5.910406
- */
- val interceptR2 = 5.791109
- val weightsR2 = Vectors.dense(3.435466, 5.910406)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.269376
+ as.numeric.data.V2. 3.736216
+ as.numeric.data.V3. 5.712356)
+ */
+ val interceptR1 = 5.269376
+ val weightsR1 = Vectors.dense(3.736216, 5.712356)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.791109
+ as.numeric.data.V2. 3.435466
+ as.numeric.data.V3. 5.910406
+ */
+ val interceptR2 = 5.791109
+ val weightsR2 = Vectors.dense(3.435466, 5.910406)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.522875
- as.numeric.data.V3. 4.214502
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.522875, 4.214502)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.263704
- as.numeric.data.V3. 4.187419
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.263704, 4.187419)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.522875
+ as.numeric.data.V3. 4.214502
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.522875, 4.214502)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.263704
+ as.numeric.data.V3. 4.187419
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.263704, 4.187419)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression with intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.324108
- as.numeric.data.V2. 3.168435
- as.numeric.data.V3. 5.200403
- */
- val interceptR1 = 5.696056
- val weightsR1 = Vectors.dense(3.670489, 6.001122)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.114723
- as.numeric.data.V2. 3.409937
- as.numeric.data.V3. 6.146531
- */
- val interceptR2 = 6.114723
- val weightsR2 = Vectors.dense(3.409937, 6.146531)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.324108
+ as.numeric.data.V2. 3.168435
+ as.numeric.data.V3. 5.200403
+ */
+ val interceptR1 = 5.696056
+ val weightsR1 = Vectors.dense(3.670489, 6.001122)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.114723
+ as.numeric.data.V2. 3.409937
+ as.numeric.data.V3. 6.146531
+ */
+ val interceptR2 = 6.114723
+ val weightsR2 = Vectors.dense(3.409937, 6.146531)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.dataM.V2. 5.673348
- as.numeric.dataM.V3. 4.322251
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.673348, 4.322251)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.477988
- as.numeric.data.V3. 4.297622
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.477988, 4.297622)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.dataM.V2. 5.673348
+ as.numeric.dataM.V3. 4.322251
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.673348, 4.322251)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.477988
+ as.numeric.data.V3. 4.297622
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.477988, 4.297622)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression model training summary") {
- val trainer = new LinearRegression
- val model = trainer.fit(dataset)
- val trainerNoPredictionCol = trainer.setPredictionCol("")
- val modelNoPredictionCol = trainerNoPredictionCol.fit(dataset)
-
-
- // Training results for the model should be available
- assert(model.hasSummary)
- assert(modelNoPredictionCol.hasSummary)
-
- // Schema should be a superset of the input dataset
- assert((dataset.schema.fieldNames.toSet + "prediction").subsetOf(
- model.summary.predictions.schema.fieldNames.toSet))
- // Validate that we re-insert a prediction column for evaluation
- val modelNoPredictionColFieldNames = modelNoPredictionCol.summary.predictions.schema.fieldNames
- assert((dataset.schema.fieldNames.toSet).subsetOf(
- modelNoPredictionColFieldNames.toSet))
- assert(modelNoPredictionColFieldNames.exists(s => s.startsWith("prediction_")))
-
- // Residuals in [[LinearRegressionResults]] should equal those manually computed
- val expectedResiduals = dataset.select("features", "label")
- .map { case Row(features: DenseVector, label: Double) =>
- val prediction =
- features(0) * model.weights(0) + features(1) * model.weights(1) + model.intercept
- label - prediction
- }
- .zip(model.summary.residuals.map(_.getDouble(0)))
- .collect()
- .foreach { case (manualResidual: Double, resultResidual: Double) =>
- assert(manualResidual ~== resultResidual relTol 1E-5)
- }
+ Seq("l-bfgs").foreach { solver => {
+ val trainer = new LinearRegression().setSolver(solver)
+ val model = trainer.fit(dataset)
+ val trainerNoPredictionCol = trainer.setPredictionCol("")
+ val modelNoPredictionCol = trainerNoPredictionCol.fit(dataset)
+
+
+ // Training results for the model should be available
+ assert(model.hasSummary)
+ assert(modelNoPredictionCol.hasSummary)
+
+ // Schema should be a superset of the input dataset
+ assert((dataset.schema.fieldNames.toSet + "prediction").subsetOf(
+ model.summary.predictions.schema.fieldNames.toSet))
+ // Validate that we re-insert a prediction column for evaluation
+ val modelNoPredictionColFieldNames
+ = modelNoPredictionCol.summary.predictions.schema.fieldNames
+ assert((dataset.schema.fieldNames.toSet).subsetOf(
+ modelNoPredictionColFieldNames.toSet))
+ assert(modelNoPredictionColFieldNames.exists(s => s.startsWith("prediction_")))
+
+ // Residuals in [[LinearRegressionResults]] should equal those manually computed
+ val expectedResiduals = dataset.select("features", "label")
+ .map { case Row(features: DenseVector, label: Double) =>
+ val prediction =
+ features(0) * model.weights(0) + features(1) * model.weights(1) + model.intercept
+ label - prediction
+ }
+ .zip(model.summary.residuals.map(_.getDouble(0)))
+ .collect()
+ .foreach { case (manualResidual: Double, resultResidual: Double) =>
+ assert(manualResidual ~== resultResidual relTol 1E-5)
+ }
- /*
- Use the following R code to generate model training results.
-
- predictions <- predict(fit, newx=features)
- residuals <- label - predictions
- > mean(residuals^2) # MSE
- [1] 0.009720325
- > mean(abs(residuals)) # MAD
- [1] 0.07863206
- > cor(predictions, label)^2# r^2
- [,1]
- s0 0.9998749
- */
- assert(model.summary.meanSquaredError ~== 0.00972035 relTol 1E-5)
- assert(model.summary.meanAbsoluteError ~== 0.07863206 relTol 1E-5)
- assert(model.summary.r2 ~== 0.9998749 relTol 1E-5)
-
- // Objective function should be monotonically decreasing for linear regression
- assert(
- model.summary
- .objectiveHistory
- .sliding(2)
- .forall(x => x(0) >= x(1)))
+ /*
+ Use the following R code to generate model training results.
+
+ predictions <- predict(fit, newx=features)
+ residuals <- label - predictions
+ > mean(residuals^2) # MSE
+ [1] 0.009720325
+ > mean(abs(residuals)) # MAD
+ [1] 0.07863206
+ > cor(predictions, label)^2# r^2
+ [,1]
+ s0 0.9998749
+ */
+ assert(model.summary.meanSquaredError ~== 0.00972035 relTol 1E-5)
+ assert(model.summary.meanAbsoluteError ~== 0.07863206 relTol 1E-5)
+ assert(model.summary.r2 ~== 0.9998749 relTol 1E-5)
+
+ // Normal solver uses "WeightedLeastSquares". This algorithm does not generate
+ // objective history because it does not run through iterations.
+ if (solver == "l-bfgs") {
+ // Objective function should be monotonically decreasing for linear regression
+ assert(
+ model.summary
+ .objectiveHistory
+ .sliding(2)
+ .forall(x => x(0) >= x(1)))
+ }
+ }}
}
test("linear regression model testset evaluation summary") {
- val trainer = new LinearRegression
- val model = trainer.fit(dataset)
-
- // Evaluating on training dataset should yield results summary equal to training summary
- val testSummary = model.evaluate(dataset)
- assert(model.summary.meanSquaredError ~== testSummary.meanSquaredError relTol 1E-5)
- assert(model.summary.r2 ~== testSummary.r2 relTol 1E-5)
- model.summary.residuals.select("residuals").collect()
- .zip(testSummary.residuals.select("residuals").collect())
- .forall { case (Row(r1: Double), Row(r2: Double)) => r1 ~== r2 relTol 1E-5 }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer = new LinearRegression().setSolver(solver)
+ val model = trainer.fit(dataset)
+
+ // Evaluating on training dataset should yield results summary equal to training summary
+ val testSummary = model.evaluate(dataset)
+ assert(model.summary.meanSquaredError ~== testSummary.meanSquaredError relTol 1E-5)
+ assert(model.summary.r2 ~== testSummary.r2 relTol 1E-5)
+ model.summary.residuals.select("residuals").collect()
+ .zip(testSummary.residuals.select("residuals").collect())
+ .forall { case (Row(r1: Double), Row(r2: Double)) => r1 ~== r2 relTol 1E-5 }
+ }}
}
test("linear regression with weighted samples"){
- val (data, weightedData) = {
- val activeData = LinearDataGenerator.generateLinearInput(
- 6.3, Array(4.7, 7.2), Array(0.9, -1.3), Array(0.7, 1.2), 500, 1, 0.1)
-
- val rnd = new Random(8392)
- val signedData = activeData.map { case p: LabeledPoint =>
- (rnd.nextGaussian() > 0.0, p)
- }
-
- val data1 = signedData.flatMap {
- case (true, p) => Iterator(p, p)
- case (false, p) => Iterator(p)
- }
-
- val weightedSignedData = signedData.flatMap {
- case (true, LabeledPoint(label, features)) =>
- Iterator(
- Instance(label, weight = 1.2, features),
- Instance(label, weight = 0.8, features)
- )
- case (false, LabeledPoint(label, features)) =>
- Iterator(
- Instance(label, weight = 0.3, features),
- Instance(label, weight = 0.1, features),
- Instance(label, weight = 0.6, features)
- )
+ Seq("auto", "l-bfgs").foreach { solver => {
--- End diff --
Does not weighted version match? Why do you skip `normal`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41899604
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,559 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }}
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
--- End diff --
Maybe you can `return` in the `if` block, or move the rest of code into `else` block.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r42402026
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,603 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with only L1 regularization case.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.24300
+ as.numeric.data.V2. 4.024821
+ as.numeric.data.V3. 6.679841
+ */
+ val interceptR1 = 6.24300
+ val weightsR1 = Vectors.dense(4.024821, 6.679841)
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.416948
+ as.numeric.data.V2. 3.893869
+ as.numeric.data.V3. 6.724286
+ */
+ val interceptR2 = 6.416948
+ val weightsR2 = Vectors.dense(3.893869, 6.724286)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression without intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.299752
- as.numeric.data.V3. 4.772913
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(6.299752, 4.772913)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.232193
- as.numeric.data.V3. 4.764229
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(6.232193, 4.764229)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with only L1 regularization case.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.299752
+ as.numeric.data.V3. 4.772913
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(6.299752, 4.772913)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.232193
+ as.numeric.data.V3. 4.764229
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(6.232193, 4.764229)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression with intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.269376
- as.numeric.data.V2. 3.736216
- as.numeric.data.V3. 5.712356)
- */
- val interceptR1 = 5.269376
- val weightsR1 = Vectors.dense(3.736216, 5.712356)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.791109
- as.numeric.data.V2. 3.435466
- as.numeric.data.V3. 5.910406
- */
- val interceptR2 = 5.791109
- val weightsR2 = Vectors.dense(3.435466, 5.910406)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.269376
+ as.numeric.data.V2. 3.736216
+ as.numeric.data.V3. 5.712356)
+ */
+ val interceptR1 = 5.269376
+ val weightsR1 = Vectors.dense(3.736216, 5.712356)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.791109
+ as.numeric.data.V2. 3.435466
+ as.numeric.data.V3. 5.910406
+ */
+ val interceptR2 = 5.791109
+ val weightsR2 = Vectors.dense(3.435466, 5.910406)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression without intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.522875
- as.numeric.data.V3. 4.214502
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.522875, 4.214502)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.263704
- as.numeric.data.V3. 4.187419
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.263704, 4.187419)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.522875
+ as.numeric.data.V3. 4.214502
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.522875, 4.214502)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.263704
+ as.numeric.data.V3. 4.187419
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.263704, 4.187419)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression with intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.324108
- as.numeric.data.V2. 3.168435
- as.numeric.data.V3. 5.200403
- */
- val interceptR1 = 5.696056
- val weightsR1 = Vectors.dense(3.670489, 6.001122)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.114723
- as.numeric.data.V2. 3.409937
- as.numeric.data.V3. 6.146531
- */
- val interceptR2 = 6.114723
- val weightsR2 = Vectors.dense(3.409937, 6.146531)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setStandardization(false).setSolver(solver)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with non-zero elasticnet parameter.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.324108
+ as.numeric.data.V2. 3.168435
+ as.numeric.data.V3. 5.200403
+ */
+ val interceptR1 = 5.696056
+ val weightsR1 = Vectors.dense(3.670489, 6.001122)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.114723
+ as.numeric.data.V2. 3.409937
+ as.numeric.data.V3. 6.146531
+ */
+ val interceptR2 = 6.114723
+ val weightsR2 = Vectors.dense(3.409937, 6.146531)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression without intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.dataM.V2. 5.673348
- as.numeric.dataM.V3. 4.322251
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.673348, 4.322251)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.477988
- as.numeric.data.V3. 4.297622
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.477988, 4.297622)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
--- End diff --
ditto
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-149141604
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-149005489
[Test build #43897 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43897/consoleFull) for PR 8884 at commit [`d996cd9`](https://github.com/apache/spark/commit/d996cd93b04895c3d69b35a894ed8accc9a1e108).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147286534
[Test build #43556 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43556/console) for PR 8884 at commit [`184d75e`](https://github.com/apache/spark/commit/184d75e5489a4672a537059bc3e84796b1412f82).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-143125360
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43007/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41811015
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,559 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }}
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.24300
+ as.numeric.data.V2. 4.024821
+ as.numeric.data.V3. 6.679841
+ */
+ val interceptR1 = 6.24300
+ val weightsR1 = Vectors.dense(4.024821, 6.679841)
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.416948
+ as.numeric.data.V2. 3.893869
+ as.numeric.data.V3. 6.724286
+ */
+ val interceptR2 = 6.416948
+ val weightsR2 = Vectors.dense(3.893869, 6.724286)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
--- End diff --
ditto
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-149007547
[Test build #43897 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43897/console) for PR 8884 at commit [`d996cd9`](https://github.com/apache/spark/commit/d996cd93b04895c3d69b35a894ed8accc9a1e108).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-148691174
[Test build #43839 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43839/console) for PR 8884 at commit [`c8d0ec9`](https://github.com/apache/spark/commit/c8d0ec9f2a96796c0ab1c9f7320985431ddcc1f1).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r42401790
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,603 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with only L1 regularization case.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+
--- End diff --
extra line. let's fix in the followup pr.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-149155400
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43921/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147286853
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43556/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40518680
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -334,13 +348,13 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
as.numeric.data.V2. 5.522875
as.numeric.data.V3. 4.214502
*/
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.522875, 4.214502)
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.522875, 4.214502)
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
- /*
+ /*
weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
--- End diff --
ditto
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142586017
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40518407
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -92,13 +93,14 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
+ Seq("auto", "l-bfgs", "normal").foreach(solver => {
--- End diff --
```
Seq("auto", "l-bfgs", "normal").foreach { solver =>
val trainer1 = new LinearRegression().setSolver(solver)
...
}
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40518531
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -113,34 +115,37 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
as.numeric.data.V2. 4.700706
as.numeric.data.V3. 7.199082
*/
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ })
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
+ Seq("auto", "l-bfgs", "normal").foreach(solver => {
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+
+ /*
--- End diff --
ditto. indentation
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-143185558
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142758222
[Test build #42929 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42929/console) for PR 8884 at commit [`5ca2a39`](https://github.com/apache/spark/commit/5ca2a3980a5a3820b125ab84481382edcf386c73).
* This patch **fails SparkR unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142820366
I think changing alpha to `0.3` is a little hacky here. The result should be the same no matter which algorithm we use, subject to some numerical differences. Could you double check the output from each algorithm? Btw, a proper way to test this is to have a string param called "solver" that defaults to "auto" but could be set explicitly to "l-bfgs" or "normal". @dbtsai
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by Lewuathe <gi...@git.apache.org>.
Github user Lewuathe commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40622089
--- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -199,8 +238,8 @@ class LinearRegression(override val uid: String)
val costFun = new LeastSquaresCostFun(instances, yStd, yMean, $(fitIntercept),
$(standardization), featuresStd, featuresMean, effectiveL2RegParam)
-
- val optimizer = if ($(elasticNetParam) == 0.0 || effectiveRegParam == 0.0) {
--- End diff --
I didn't know that. Thank you so much for sharing!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40518599
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -190,13 +198,12 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
as.numeric.data.V2. 4.024821
as.numeric.data.V3. 6.679841
*/
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
+ val interceptR1 = 6.24300
+ val weightsR1 = Vectors.dense(4.024821, 6.679841)
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
+ /*
--- End diff --
ditto.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142748457
test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40518881
--- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -140,6 +141,17 @@ class LinearRegression(override val uid: String)
def setWeightCol(value: String): this.type = set(weightCol, value)
setDefault(weightCol -> "")
+ /**
+ * Set the solver algorithm for optimization.
+ * In case of linear regression, this can be "l-bfgs", "normal" and "auto".
+ * The default value is "auto" which means that the solver algorithm is
+ * selected automatically.
+ * @param value
+ * @return
--- End diff --
remove `@return`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40569761
--- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -199,8 +238,8 @@ class LinearRegression(override val uid: String)
val costFun = new LeastSquaresCostFun(instances, yStd, yMean, $(fitIntercept),
$(standardization), featuresStd, featuresMean, effectiveL2RegParam)
-
- val optimizer = if ($(elasticNetParam) == 0.0 || effectiveRegParam == 0.0) {
--- End diff --
Owlqn is an generalized version of lbfgs but can solve L1. So when the solver is lbfgs, and L1 is used, we should use owlqn.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147281654
[Test build #43556 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43556/consoleFull) for PR 8884 at commit [`184d75e`](https://github.com/apache/spark/commit/184d75e5489a4672a537059bc3e84796b1412f82).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-145968799
@Lewuathe What's the current status of this? We would like to have it in Spark 1.6. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-143125359
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40518699
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -350,28 +364,31 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
as.numeric.data.V2. 5.263704
as.numeric.data.V3. 4.187419
*/
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.263704, 4.187419)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.263704, 4.187419)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ })
}
test("linear regression with intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
+ Seq("auto").foreach(solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
--- End diff --
ditto. check if the exception is thrown when the solver is not supported.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147910824
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-149141582
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-149005126
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-149145144
**[Test build #43921 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43921/consoleFull)** for PR 8884 at commit [`c121e10`](https://github.com/apache/spark/commit/c121e10813b02be90c7ff695a98976a57589b7f6).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147181340
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40518703
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -350,28 +364,31 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
as.numeric.data.V2. 5.263704
as.numeric.data.V3. 4.187419
*/
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.263704, 4.187419)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.263704, 4.187419)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ })
}
test("linear regression with intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
+ Seq("auto").foreach(solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
--- End diff --
ditto. indentation
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40518479
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -113,34 +115,37 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
as.numeric.data.V2. 4.700706
as.numeric.data.V3. 7.199082
*/
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
--- End diff --
Remove the extra line.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147182510
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by Lewuathe <gi...@git.apache.org>.
Github user Lewuathe commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142823694
WeightedLeastSquare does not create a summary history. So in case of using this algorithm we cannot confirm the existence of summary. In order to make sure to use LBFGS, I changed elastic net parameter. I agree with you, this is hacky.
If we can set a solver parameter at test explicitly, it might be better.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147182247
[Test build #43541 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43541/consoleFull) for PR 8884 at commit [`99916cc`](https://github.com/apache/spark/commit/99916cc1ade4202ba32862aa11b6c2f42f7daa2b).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40518653
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -206,30 +213,32 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
as.numeric.data.V2. 3.893869
as.numeric.data.V3. 6.724286
*/
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
+ val interceptR2 = 6.416948
+ val weightsR2 = Vectors.dense(3.893869, 6.724286)
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ })
}
test("linear regression without intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
+ Seq("auto").foreach(solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
--- End diff --
ditto
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-143120484
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142749130
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r42339518
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +96,641 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with only L1 regularization case.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ break
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+ }
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.24300
+ as.numeric.data.V2. 4.024821
+ as.numeric.data.V3. 6.679841
+ */
+ val interceptR1 = 6.24300
+ val weightsR1 = Vectors.dense(4.024821, 6.679841)
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.416948
+ as.numeric.data.V2. 3.893869
+ as.numeric.data.V3. 6.724286
+ */
+ val interceptR2 = 6.416948
+ val weightsR2 = Vectors.dense(3.893869, 6.724286)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression without intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.299752
- as.numeric.data.V3. 4.772913
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(6.299752, 4.772913)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.232193
- as.numeric.data.V3. 4.764229
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(6.232193, 4.764229)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with only L1 regularization case.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ break
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+ }
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.299752
+ as.numeric.data.V3. 4.772913
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(6.299752, 4.772913)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.232193
+ as.numeric.data.V3. 4.764229
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(6.232193, 4.764229)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression with intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.269376
- as.numeric.data.V2. 3.736216
- as.numeric.data.V3. 5.712356)
- */
- val interceptR1 = 5.269376
- val weightsR1 = Vectors.dense(3.736216, 5.712356)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.791109
- as.numeric.data.V2. 3.435466
- as.numeric.data.V3. 5.910406
- */
- val interceptR2 = 5.791109
- val weightsR2 = Vectors.dense(3.435466, 5.910406)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.269376
+ as.numeric.data.V2. 3.736216
+ as.numeric.data.V3. 5.712356)
+ */
+ val interceptR1 = 5.269376
+ val weightsR1 = Vectors.dense(3.736216, 5.712356)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.791109
+ as.numeric.data.V2. 3.435466
+ as.numeric.data.V3. 5.910406
+ */
+ val interceptR2 = 5.791109
+ val weightsR2 = Vectors.dense(3.435466, 5.910406)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression without intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.522875
- as.numeric.data.V3. 4.214502
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.522875, 4.214502)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.263704
- as.numeric.data.V3. 4.187419
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.263704, 4.187419)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.522875
+ as.numeric.data.V3. 4.214502
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.522875, 4.214502)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.263704
+ as.numeric.data.V3. 4.187419
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.263704, 4.187419)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression with intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.324108
- as.numeric.data.V2. 3.168435
- as.numeric.data.V3. 5.200403
- */
- val interceptR1 = 5.696056
- val weightsR1 = Vectors.dense(3.670489, 6.001122)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.114723
- as.numeric.data.V2. 3.409937
- as.numeric.data.V3. 6.146531
- */
- val interceptR2 = 6.114723
- val weightsR2 = Vectors.dense(3.409937, 6.146531)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setStandardization(false).setSolver(solver)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with non-zero elasticnet parameter.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ break
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+ }
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.324108
+ as.numeric.data.V2. 3.168435
+ as.numeric.data.V3. 5.200403
+ */
+ val interceptR1 = 5.696056
+ val weightsR1 = Vectors.dense(3.670489, 6.001122)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.114723
+ as.numeric.data.V2. 3.409937
+ as.numeric.data.V3. 6.146531
+ */
+ val interceptR2 = 6.114723
+ val weightsR2 = Vectors.dense(3.409937, 6.146531)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression without intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.dataM.V2. 5.673348
- as.numeric.dataM.V3. 4.322251
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.673348, 4.322251)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.477988
- as.numeric.data.V3. 4.297622
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.477988, 4.297622)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with non-zero elasticnet parameter.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ break
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+ }
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.dataM.V2. 5.673348
+ as.numeric.dataM.V3. 4.322251
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.673348, 4.322251)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.477988
+ as.numeric.data.V3. 4.297622
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.477988, 4.297622)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression model training summary") {
- val trainer = new LinearRegression
- val model = trainer.fit(dataset)
- val trainerNoPredictionCol = trainer.setPredictionCol("")
- val modelNoPredictionCol = trainerNoPredictionCol.fit(dataset)
-
-
- // Training results for the model should be available
- assert(model.hasSummary)
- assert(modelNoPredictionCol.hasSummary)
-
- // Schema should be a superset of the input dataset
- assert((dataset.schema.fieldNames.toSet + "prediction").subsetOf(
- model.summary.predictions.schema.fieldNames.toSet))
- // Validate that we re-insert a prediction column for evaluation
- val modelNoPredictionColFieldNames = modelNoPredictionCol.summary.predictions.schema.fieldNames
- assert((dataset.schema.fieldNames.toSet).subsetOf(
- modelNoPredictionColFieldNames.toSet))
- assert(modelNoPredictionColFieldNames.exists(s => s.startsWith("prediction_")))
-
- // Residuals in [[LinearRegressionResults]] should equal those manually computed
- val expectedResiduals = dataset.select("features", "label")
- .map { case Row(features: DenseVector, label: Double) =>
- val prediction =
- features(0) * model.weights(0) + features(1) * model.weights(1) + model.intercept
- label - prediction
- }
- .zip(model.summary.residuals.map(_.getDouble(0)))
- .collect()
- .foreach { case (manualResidual: Double, resultResidual: Double) =>
- assert(manualResidual ~== resultResidual relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer = new LinearRegression().setSolver(solver)
+ val model = trainer.fit(dataset)
+ val trainerNoPredictionCol = trainer.setPredictionCol("")
+ val modelNoPredictionCol = trainerNoPredictionCol.fit(dataset)
+
+
+ // Training results for the model should be available
+ assert(model.hasSummary)
+ assert(modelNoPredictionCol.hasSummary)
+
+ // Schema should be a superset of the input dataset
+ assert((dataset.schema.fieldNames.toSet + "prediction").subsetOf(
+ model.summary.predictions.schema.fieldNames.toSet))
+ // Validate that we re-insert a prediction column for evaluation
+ val modelNoPredictionColFieldNames
+ = modelNoPredictionCol.summary.predictions.schema.fieldNames
+ assert((dataset.schema.fieldNames.toSet).subsetOf(
+ modelNoPredictionColFieldNames.toSet))
+ assert(modelNoPredictionColFieldNames.exists(s => s.startsWith("prediction_")))
+
+ // Residuals in [[LinearRegressionResults]] should equal those manually computed
+ val expectedResiduals = dataset.select("features", "label")
+ .map { case Row(features: DenseVector, label: Double) =>
+ val prediction =
+ features(0) * model.weights(0) + features(1) * model.weights(1) + model.intercept
+ label - prediction
+ }
+ .zip(model.summary.residuals.map(_.getDouble(0)))
+ .collect()
+ .foreach { case (manualResidual: Double, resultResidual: Double) =>
+ assert(manualResidual ~== resultResidual relTol 1E-5)
+ }
- /*
- Use the following R code to generate model training results.
-
- predictions <- predict(fit, newx=features)
- residuals <- label - predictions
- > mean(residuals^2) # MSE
- [1] 0.009720325
- > mean(abs(residuals)) # MAD
- [1] 0.07863206
- > cor(predictions, label)^2# r^2
- [,1]
- s0 0.9998749
- */
- assert(model.summary.meanSquaredError ~== 0.00972035 relTol 1E-5)
- assert(model.summary.meanAbsoluteError ~== 0.07863206 relTol 1E-5)
- assert(model.summary.r2 ~== 0.9998749 relTol 1E-5)
-
- // Objective function should be monotonically decreasing for linear regression
- assert(
- model.summary
- .objectiveHistory
- .sliding(2)
- .forall(x => x(0) >= x(1)))
+ /*
+ Use the following R code to generate model training results.
+
+ predictions <- predict(fit, newx=features)
+ residuals <- label - predictions
+ > mean(residuals^2) # MSE
+ [1] 0.009720325
+ > mean(abs(residuals)) # MAD
+ [1] 0.07863206
+ > cor(predictions, label)^2# r^2
+ [,1]
+ s0 0.9998749
+ */
+ assert(model.summary.meanSquaredError ~== 0.00972035 relTol 1E-5)
+ assert(model.summary.meanAbsoluteError ~== 0.07863206 relTol 1E-5)
+ assert(model.summary.r2 ~== 0.9998749 relTol 1E-5)
+
+ // Normal solver uses "WeightedLeastSquares". This algorithm does not generate
+ // objective history because it does not run through iterations.
+ if (solver == "l-bfgs") {
+ // Objective function should be monotonically decreasing for linear regression
+ assert(
+ model.summary
+ .objectiveHistory
+ .sliding(2)
+ .forall(x => x(0) >= x(1)))
+ }
+ }
}
test("linear regression model testset evaluation summary") {
- val trainer = new LinearRegression
- val model = trainer.fit(dataset)
-
- // Evaluating on training dataset should yield results summary equal to training summary
- val testSummary = model.evaluate(dataset)
- assert(model.summary.meanSquaredError ~== testSummary.meanSquaredError relTol 1E-5)
- assert(model.summary.r2 ~== testSummary.r2 relTol 1E-5)
- model.summary.residuals.select("residuals").collect()
- .zip(testSummary.residuals.select("residuals").collect())
- .forall { case (Row(r1: Double), Row(r2: Double)) => r1 ~== r2 relTol 1E-5 }
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer = new LinearRegression().setSolver(solver)
+ val model = trainer.fit(dataset)
+
+ // Evaluating on training dataset should yield results summary equal to training summary
+ val testSummary = model.evaluate(dataset)
+ assert(model.summary.meanSquaredError ~== testSummary.meanSquaredError relTol 1E-5)
+ assert(model.summary.r2 ~== testSummary.r2 relTol 1E-5)
+ model.summary.residuals.select("residuals").collect()
+ .zip(testSummary.residuals.select("residuals").collect())
+ .forall { case (Row(r1: Double), Row(r2: Double)) => r1 ~== r2 relTol 1E-5 }
+ }
}
test("linear regression with weighted samples"){
- val (data, weightedData) = {
- val activeData = LinearDataGenerator.generateLinearInput(
- 6.3, Array(4.7, 7.2), Array(0.9, -1.3), Array(0.7, 1.2), 500, 1, 0.1)
-
- val rnd = new Random(8392)
- val signedData = activeData.map { case p: LabeledPoint =>
- (rnd.nextGaussian() > 0.0, p)
- }
-
- val data1 = signedData.flatMap {
- case (true, p) => Iterator(p, p)
- case (false, p) => Iterator(p)
- }
-
- val weightedSignedData = signedData.flatMap {
- case (true, LabeledPoint(label, features)) =>
- Iterator(
- Instance(label, weight = 1.2, features),
- Instance(label, weight = 0.8, features)
- )
- case (false, LabeledPoint(label, features)) =>
- Iterator(
- Instance(label, weight = 0.3, features),
- Instance(label, weight = 0.1, features),
- Instance(label, weight = 0.6, features)
- )
- }
-
- val noiseData = LinearDataGenerator.generateLinearInput(
- 2, Array(1, 3), Array(0.9, -1.3), Array(0.7, 1.2), 500, 1, 0.1)
- val weightedNoiseData = noiseData.map {
- case LabeledPoint(label, features) => Instance(label, weight = 0, features)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val (data, weightedData) = {
+ val activeData = LinearDataGenerator.generateLinearInput(
+ 6.3, Array(4.7, 7.2), Array(0.9, -1.3), Array(0.7, 1.2), 500, 1, 0.1)
+
+ val rnd = new Random(8392)
+ val signedData = activeData.map { case p: LabeledPoint =>
+ (rnd.nextGaussian() > 0.0, p)
+ }
+
+ val data1 = signedData.flatMap {
+ case (true, p) => Iterator(p, p)
+ case (false, p) => Iterator(p)
+ }
+
+ val weightedSignedData = signedData.flatMap {
+ case (true, LabeledPoint(label, features)) =>
+ Iterator(
+ Instance(label, weight = 1.2, features),
+ Instance(label, weight = 0.8, features)
+ )
+ case (false, LabeledPoint(label, features)) =>
+ Iterator(
+ Instance(label, weight = 0.3, features),
+ Instance(label, weight = 0.1, features),
+ Instance(label, weight = 0.6, features)
+ )
+ }
+
+ val noiseData = LinearDataGenerator.generateLinearInput(
+ 2, Array(1, 3), Array(0.9, -1.3), Array(0.7, 1.2), 500, 1, 0.1)
+ val weightedNoiseData = noiseData.map {
+ case LabeledPoint(label, features) => Instance(label, weight = 0, features)
+ }
+ val data2 = weightedSignedData ++ weightedNoiseData
+
+ (sqlContext.createDataFrame(sc.parallelize(data1, 4)),
+ sqlContext.createDataFrame(sc.parallelize(data2, 4)))
+ }
+
+ val trainer1a = (new LinearRegression).setFitIntercept(true)
+ .setElasticNetParam(0.38).setRegParam(0.21).setStandardization(true).setSolver(solver)
+ val trainer1b = (new LinearRegression).setFitIntercept(true).setWeightCol("weight")
+ .setElasticNetParam(0.38).setRegParam(0.21).setStandardization(true).setSolver(solver)
+
+ var model1a0: LinearRegressionModel = null
+ var model1a1: LinearRegressionModel = null
+ var model1b: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with non-zero elasticnet parameter.
+ if (solver == "normal") {
--- End diff --
Yes, either change it to 0.0 or have a copy of it. I think change it to 0.0 should be fine.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-144388721
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r42401980
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,603 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with only L1 regularization case.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.24300
+ as.numeric.data.V2. 4.024821
+ as.numeric.data.V3. 6.679841
+ */
+ val interceptR1 = 6.24300
+ val weightsR1 = Vectors.dense(4.024821, 6.679841)
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.416948
+ as.numeric.data.V2. 3.893869
+ as.numeric.data.V3. 6.724286
+ */
+ val interceptR2 = 6.416948
+ val weightsR2 = Vectors.dense(3.893869, 6.724286)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression without intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.299752
- as.numeric.data.V3. 4.772913
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(6.299752, 4.772913)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.232193
- as.numeric.data.V3. 4.764229
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(6.232193, 4.764229)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with only L1 regularization case.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.299752
+ as.numeric.data.V3. 4.772913
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(6.299752, 4.772913)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.232193
+ as.numeric.data.V3. 4.764229
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(6.232193, 4.764229)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression with intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.269376
- as.numeric.data.V2. 3.736216
- as.numeric.data.V3. 5.712356)
- */
- val interceptR1 = 5.269376
- val weightsR1 = Vectors.dense(3.736216, 5.712356)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.791109
- as.numeric.data.V2. 3.435466
- as.numeric.data.V3. 5.910406
- */
- val interceptR2 = 5.791109
- val weightsR2 = Vectors.dense(3.435466, 5.910406)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.269376
+ as.numeric.data.V2. 3.736216
+ as.numeric.data.V3. 5.712356)
+ */
+ val interceptR1 = 5.269376
+ val weightsR1 = Vectors.dense(3.736216, 5.712356)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.791109
+ as.numeric.data.V2. 3.435466
+ as.numeric.data.V3. 5.910406
+ */
+ val interceptR2 = 5.791109
+ val weightsR2 = Vectors.dense(3.435466, 5.910406)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression without intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.522875
- as.numeric.data.V3. 4.214502
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.522875, 4.214502)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.263704
- as.numeric.data.V3. 4.187419
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.263704, 4.187419)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.522875
+ as.numeric.data.V3. 4.214502
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.522875, 4.214502)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.263704
+ as.numeric.data.V3. 4.187419
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.263704, 4.187419)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression with intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.324108
- as.numeric.data.V2. 3.168435
- as.numeric.data.V3. 5.200403
- */
- val interceptR1 = 5.696056
- val weightsR1 = Vectors.dense(3.670489, 6.001122)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.114723
- as.numeric.data.V2. 3.409937
- as.numeric.data.V3. 6.146531
- */
- val interceptR2 = 6.114723
- val weightsR2 = Vectors.dense(3.409937, 6.146531)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setStandardization(false).setSolver(solver)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
--- End diff --
ditto.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by Lewuathe <gi...@git.apache.org>.
Github user Lewuathe commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41856633
--- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -130,9 +131,54 @@ class LinearRegression(override val uid: String)
def setWeightCol(value: String): this.type = set(weightCol, value)
setDefault(weightCol -> "")
+ /**
+ * Set the solver algorithm used for optimization.
+ * In case of linear regression, this can be "l-bfgs", "normal" and "auto".
+ * The default value is "auto" which means that the solver algorithm is
+ * selected automatically.
+ * @group setParam
+ */
+ def setSolver(value: String): this.type = set(solver, value)
+ setDefault(solver -> "auto")
+
override protected def train(dataset: DataFrame): LinearRegressionModel = {
+ // Extract the number of features before deciding optimization solver.
+ val numFeatures = dataset.select(col($(featuresCol))).limit(1).map {
+ case Row(features: Vector) =>
+ features.size
+ }.toArray()(0)
// Extract columns from data. If dataset is persisted, do not persist instances.
val w = if ($(weightCol).isEmpty) lit(1.0) else col($(weightCol))
+
+ if ($(solver) == "normal" || ($(solver) == "auto"
+ && $(elasticNetParam) == 0.0 && numFeatures <= 4096)) {
+ require($(elasticNetParam) == 0.0, "Only L2 regularization can be used when normal " +
+ "solver is selected.'")
+ // In case of feature size is small, WeightedLeastSquares can train more efficiently
+ // because it requires one pass through to the data. (SPARK-10668)
+ val instances: RDD[WeightedLeastSquares.Instance] = dataset.select(
+ col($(labelCol)), w, col($(featuresCol))).map {
+ case Row(label: Double, weight: Double, features: Vector) =>
+ WeightedLeastSquares.Instance(weight, features, label)
+ }
+
+ val optimizer = new WeightedLeastSquares($(fitIntercept), $(regParam),
+ $(standardization), true)
+ val model = optimizer.fit(instances)
+ // When it is trained by WeightedLeastSquares, training summary does not
+ // attached returned model.
+ val lrModel = new LinearRegressionModel(uid, model.coefficients, model.intercept)
--- End diff --
The finally returned value is copied at L.177. Is it also necessary to copy here?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41810020
--- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -130,9 +131,54 @@ class LinearRegression(override val uid: String)
def setWeightCol(value: String): this.type = set(weightCol, value)
setDefault(weightCol -> "")
+ /**
+ * Set the solver algorithm used for optimization.
+ * In case of linear regression, this can be "l-bfgs", "normal" and "auto".
+ * The default value is "auto" which means that the solver algorithm is
+ * selected automatically.
+ * @group setParam
+ */
+ def setSolver(value: String): this.type = set(solver, value)
+ setDefault(solver -> "auto")
+
override protected def train(dataset: DataFrame): LinearRegressionModel = {
+ // Extract the number of features before deciding optimization solver.
+ val numFeatures = dataset.select(col($(featuresCol))).limit(1).map {
+ case Row(features: Vector) =>
+ features.size
+ }.toArray()(0)
// Extract columns from data. If dataset is persisted, do not persist instances.
val w = if ($(weightCol).isEmpty) lit(1.0) else col($(weightCol))
+
+ if ($(solver) == "normal" || ($(solver) == "auto"
+ && $(elasticNetParam) == 0.0 && numFeatures <= 4096)) {
+ require($(elasticNetParam) == 0.0, "Only L2 regularization can be used when normal " +
+ "solver is selected.'")
+ // In case of feature size is small, WeightedLeastSquares can train more efficiently
+ // because it requires one pass through to the data. (SPARK-10668)
+ val instances: RDD[WeightedLeastSquares.Instance] = dataset.select(
+ col($(labelCol)), w, col($(featuresCol))).map {
+ case Row(label: Double, weight: Double, features: Vector) =>
+ WeightedLeastSquares.Instance(weight, features, label)
+ }
+
+ val optimizer = new WeightedLeastSquares($(fitIntercept), $(regParam),
+ $(standardization), true)
+ val model = optimizer.fit(instances)
+ // When it is trained by WeightedLeastSquares, training summary does not
+ // attached returned model.
+ val lrModel = new LinearRegressionModel(uid, model.coefficients, model.intercept)
--- End diff --
Do `copyValues` here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r42327619
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +96,641 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with only L1 regularization case.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ break
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+ }
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.24300
+ as.numeric.data.V2. 4.024821
+ as.numeric.data.V3. 6.679841
+ */
+ val interceptR1 = 6.24300
+ val weightsR1 = Vectors.dense(4.024821, 6.679841)
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.416948
+ as.numeric.data.V2. 3.893869
+ as.numeric.data.V3. 6.724286
+ */
+ val interceptR2 = 6.416948
+ val weightsR2 = Vectors.dense(3.893869, 6.724286)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression without intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.299752
- as.numeric.data.V3. 4.772913
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(6.299752, 4.772913)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.232193
- as.numeric.data.V3. 4.764229
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(6.232193, 4.764229)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with only L1 regularization case.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ break
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+ }
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.299752
+ as.numeric.data.V3. 4.772913
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(6.299752, 4.772913)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.232193
+ as.numeric.data.V3. 4.764229
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(6.232193, 4.764229)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression with intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.269376
- as.numeric.data.V2. 3.736216
- as.numeric.data.V3. 5.712356)
- */
- val interceptR1 = 5.269376
- val weightsR1 = Vectors.dense(3.736216, 5.712356)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.791109
- as.numeric.data.V2. 3.435466
- as.numeric.data.V3. 5.910406
- */
- val interceptR2 = 5.791109
- val weightsR2 = Vectors.dense(3.435466, 5.910406)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.269376
+ as.numeric.data.V2. 3.736216
+ as.numeric.data.V3. 5.712356)
+ */
+ val interceptR1 = 5.269376
+ val weightsR1 = Vectors.dense(3.736216, 5.712356)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.791109
+ as.numeric.data.V2. 3.435466
+ as.numeric.data.V3. 5.910406
+ */
+ val interceptR2 = 5.791109
+ val weightsR2 = Vectors.dense(3.435466, 5.910406)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression without intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.522875
- as.numeric.data.V3. 4.214502
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.522875, 4.214502)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.263704
- as.numeric.data.V3. 4.187419
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.263704, 4.187419)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.522875
+ as.numeric.data.V3. 4.214502
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.522875, 4.214502)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.263704
+ as.numeric.data.V3. 4.187419
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.263704, 4.187419)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression with intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.324108
- as.numeric.data.V2. 3.168435
- as.numeric.data.V3. 5.200403
- */
- val interceptR1 = 5.696056
- val weightsR1 = Vectors.dense(3.670489, 6.001122)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.114723
- as.numeric.data.V2. 3.409937
- as.numeric.data.V3. 6.146531
- */
- val interceptR2 = 6.114723
- val weightsR2 = Vectors.dense(3.409937, 6.146531)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setStandardization(false).setSolver(solver)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with non-zero elasticnet parameter.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ break
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+ }
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.324108
+ as.numeric.data.V2. 3.168435
+ as.numeric.data.V3. 5.200403
+ */
+ val interceptR1 = 5.696056
+ val weightsR1 = Vectors.dense(3.670489, 6.001122)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.114723
+ as.numeric.data.V2. 3.409937
+ as.numeric.data.V3. 6.146531
+ */
+ val interceptR2 = 6.114723
+ val weightsR2 = Vectors.dense(3.409937, 6.146531)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression without intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.dataM.V2. 5.673348
- as.numeric.dataM.V3. 4.322251
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.673348, 4.322251)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.477988
- as.numeric.data.V3. 4.297622
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.477988, 4.297622)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with non-zero elasticnet parameter.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ break
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+ }
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.dataM.V2. 5.673348
+ as.numeric.dataM.V3. 4.322251
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.673348, 4.322251)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.477988
+ as.numeric.data.V3. 4.297622
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.477988, 4.297622)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression model training summary") {
- val trainer = new LinearRegression
- val model = trainer.fit(dataset)
- val trainerNoPredictionCol = trainer.setPredictionCol("")
- val modelNoPredictionCol = trainerNoPredictionCol.fit(dataset)
-
-
- // Training results for the model should be available
- assert(model.hasSummary)
- assert(modelNoPredictionCol.hasSummary)
-
- // Schema should be a superset of the input dataset
- assert((dataset.schema.fieldNames.toSet + "prediction").subsetOf(
- model.summary.predictions.schema.fieldNames.toSet))
- // Validate that we re-insert a prediction column for evaluation
- val modelNoPredictionColFieldNames = modelNoPredictionCol.summary.predictions.schema.fieldNames
- assert((dataset.schema.fieldNames.toSet).subsetOf(
- modelNoPredictionColFieldNames.toSet))
- assert(modelNoPredictionColFieldNames.exists(s => s.startsWith("prediction_")))
-
- // Residuals in [[LinearRegressionResults]] should equal those manually computed
- val expectedResiduals = dataset.select("features", "label")
- .map { case Row(features: DenseVector, label: Double) =>
- val prediction =
- features(0) * model.weights(0) + features(1) * model.weights(1) + model.intercept
- label - prediction
- }
- .zip(model.summary.residuals.map(_.getDouble(0)))
- .collect()
- .foreach { case (manualResidual: Double, resultResidual: Double) =>
- assert(manualResidual ~== resultResidual relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer = new LinearRegression().setSolver(solver)
+ val model = trainer.fit(dataset)
+ val trainerNoPredictionCol = trainer.setPredictionCol("")
+ val modelNoPredictionCol = trainerNoPredictionCol.fit(dataset)
+
+
+ // Training results for the model should be available
+ assert(model.hasSummary)
+ assert(modelNoPredictionCol.hasSummary)
+
+ // Schema should be a superset of the input dataset
+ assert((dataset.schema.fieldNames.toSet + "prediction").subsetOf(
+ model.summary.predictions.schema.fieldNames.toSet))
+ // Validate that we re-insert a prediction column for evaluation
+ val modelNoPredictionColFieldNames
+ = modelNoPredictionCol.summary.predictions.schema.fieldNames
+ assert((dataset.schema.fieldNames.toSet).subsetOf(
+ modelNoPredictionColFieldNames.toSet))
+ assert(modelNoPredictionColFieldNames.exists(s => s.startsWith("prediction_")))
+
+ // Residuals in [[LinearRegressionResults]] should equal those manually computed
+ val expectedResiduals = dataset.select("features", "label")
+ .map { case Row(features: DenseVector, label: Double) =>
+ val prediction =
+ features(0) * model.weights(0) + features(1) * model.weights(1) + model.intercept
+ label - prediction
+ }
+ .zip(model.summary.residuals.map(_.getDouble(0)))
+ .collect()
+ .foreach { case (manualResidual: Double, resultResidual: Double) =>
+ assert(manualResidual ~== resultResidual relTol 1E-5)
+ }
- /*
- Use the following R code to generate model training results.
-
- predictions <- predict(fit, newx=features)
- residuals <- label - predictions
- > mean(residuals^2) # MSE
- [1] 0.009720325
- > mean(abs(residuals)) # MAD
- [1] 0.07863206
- > cor(predictions, label)^2# r^2
- [,1]
- s0 0.9998749
- */
- assert(model.summary.meanSquaredError ~== 0.00972035 relTol 1E-5)
- assert(model.summary.meanAbsoluteError ~== 0.07863206 relTol 1E-5)
- assert(model.summary.r2 ~== 0.9998749 relTol 1E-5)
-
- // Objective function should be monotonically decreasing for linear regression
- assert(
- model.summary
- .objectiveHistory
- .sliding(2)
- .forall(x => x(0) >= x(1)))
+ /*
+ Use the following R code to generate model training results.
+
+ predictions <- predict(fit, newx=features)
+ residuals <- label - predictions
+ > mean(residuals^2) # MSE
+ [1] 0.009720325
+ > mean(abs(residuals)) # MAD
+ [1] 0.07863206
+ > cor(predictions, label)^2# r^2
+ [,1]
+ s0 0.9998749
+ */
+ assert(model.summary.meanSquaredError ~== 0.00972035 relTol 1E-5)
+ assert(model.summary.meanAbsoluteError ~== 0.07863206 relTol 1E-5)
+ assert(model.summary.r2 ~== 0.9998749 relTol 1E-5)
+
+ // Normal solver uses "WeightedLeastSquares". This algorithm does not generate
+ // objective history because it does not run through iterations.
+ if (solver == "l-bfgs") {
+ // Objective function should be monotonically decreasing for linear regression
+ assert(
+ model.summary
+ .objectiveHistory
+ .sliding(2)
+ .forall(x => x(0) >= x(1)))
+ }
+ }
}
test("linear regression model testset evaluation summary") {
- val trainer = new LinearRegression
- val model = trainer.fit(dataset)
-
- // Evaluating on training dataset should yield results summary equal to training summary
- val testSummary = model.evaluate(dataset)
- assert(model.summary.meanSquaredError ~== testSummary.meanSquaredError relTol 1E-5)
- assert(model.summary.r2 ~== testSummary.r2 relTol 1E-5)
- model.summary.residuals.select("residuals").collect()
- .zip(testSummary.residuals.select("residuals").collect())
- .forall { case (Row(r1: Double), Row(r2: Double)) => r1 ~== r2 relTol 1E-5 }
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer = new LinearRegression().setSolver(solver)
+ val model = trainer.fit(dataset)
+
+ // Evaluating on training dataset should yield results summary equal to training summary
+ val testSummary = model.evaluate(dataset)
+ assert(model.summary.meanSquaredError ~== testSummary.meanSquaredError relTol 1E-5)
+ assert(model.summary.r2 ~== testSummary.r2 relTol 1E-5)
+ model.summary.residuals.select("residuals").collect()
+ .zip(testSummary.residuals.select("residuals").collect())
+ .forall { case (Row(r1: Double), Row(r2: Double)) => r1 ~== r2 relTol 1E-5 }
+ }
}
test("linear regression with weighted samples"){
- val (data, weightedData) = {
- val activeData = LinearDataGenerator.generateLinearInput(
- 6.3, Array(4.7, 7.2), Array(0.9, -1.3), Array(0.7, 1.2), 500, 1, 0.1)
-
- val rnd = new Random(8392)
- val signedData = activeData.map { case p: LabeledPoint =>
- (rnd.nextGaussian() > 0.0, p)
- }
-
- val data1 = signedData.flatMap {
- case (true, p) => Iterator(p, p)
- case (false, p) => Iterator(p)
- }
-
- val weightedSignedData = signedData.flatMap {
- case (true, LabeledPoint(label, features)) =>
- Iterator(
- Instance(label, weight = 1.2, features),
- Instance(label, weight = 0.8, features)
- )
- case (false, LabeledPoint(label, features)) =>
- Iterator(
- Instance(label, weight = 0.3, features),
- Instance(label, weight = 0.1, features),
- Instance(label, weight = 0.6, features)
- )
- }
-
- val noiseData = LinearDataGenerator.generateLinearInput(
- 2, Array(1, 3), Array(0.9, -1.3), Array(0.7, 1.2), 500, 1, 0.1)
- val weightedNoiseData = noiseData.map {
- case LabeledPoint(label, features) => Instance(label, weight = 0, features)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val (data, weightedData) = {
+ val activeData = LinearDataGenerator.generateLinearInput(
+ 6.3, Array(4.7, 7.2), Array(0.9, -1.3), Array(0.7, 1.2), 500, 1, 0.1)
+
+ val rnd = new Random(8392)
+ val signedData = activeData.map { case p: LabeledPoint =>
+ (rnd.nextGaussian() > 0.0, p)
+ }
+
+ val data1 = signedData.flatMap {
+ case (true, p) => Iterator(p, p)
+ case (false, p) => Iterator(p)
+ }
+
+ val weightedSignedData = signedData.flatMap {
+ case (true, LabeledPoint(label, features)) =>
+ Iterator(
+ Instance(label, weight = 1.2, features),
+ Instance(label, weight = 0.8, features)
+ )
+ case (false, LabeledPoint(label, features)) =>
+ Iterator(
+ Instance(label, weight = 0.3, features),
+ Instance(label, weight = 0.1, features),
+ Instance(label, weight = 0.6, features)
+ )
+ }
+
+ val noiseData = LinearDataGenerator.generateLinearInput(
+ 2, Array(1, 3), Array(0.9, -1.3), Array(0.7, 1.2), 500, 1, 0.1)
+ val weightedNoiseData = noiseData.map {
+ case LabeledPoint(label, features) => Instance(label, weight = 0, features)
+ }
+ val data2 = weightedSignedData ++ weightedNoiseData
+
+ (sqlContext.createDataFrame(sc.parallelize(data1, 4)),
+ sqlContext.createDataFrame(sc.parallelize(data2, 4)))
+ }
+
+ val trainer1a = (new LinearRegression).setFitIntercept(true)
+ .setElasticNetParam(0.38).setRegParam(0.21).setStandardization(true).setSolver(solver)
+ val trainer1b = (new LinearRegression).setFitIntercept(true).setWeightCol("weight")
+ .setElasticNetParam(0.38).setRegParam(0.21).setStandardization(true).setSolver(solver)
+
+ var model1a0: LinearRegressionModel = null
+ var model1a1: LinearRegressionModel = null
+ var model1b: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with non-zero elasticnet parameter.
+ if (solver == "normal") {
--- End diff --
Here is the main issue for me. When `elasticNetParam == 0.0`, weighted sample LiRs should have the same solution with auto, normal and l-bfgs which is not tested.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by Lewuathe <gi...@git.apache.org>.
Github user Lewuathe commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-149691655
@dbtsai Sorry. I forgot to link here.
https://issues.apache.org/jira/plugins/servlet/mobile#issue/SPARK-11207
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147181342
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142597885
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40518860
--- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -140,6 +141,17 @@ class LinearRegression(override val uid: String)
def setWeightCol(value: String): this.type = set(weightCol, value)
setDefault(weightCol -> "")
+ /**
+ * Set the solver algorithm for optimization.
--- End diff --
Set the solver algorithm used for optimization.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-148679490
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147184276
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142783213
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142783937
[Test build #42950 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42950/consoleFull) for PR 8884 at commit [`fa01cae`](https://github.com/apache/spark/commit/fa01cae9e7358b932cdab7d7b3d07fa5a28c44fb).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/8884
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-149005130
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by yanboliang <gi...@git.apache.org>.
Github user yanboliang commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40528592
--- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -169,6 +181,33 @@ class LinearRegression(override val uid: String)
val yMean = ySummarizer.mean(0)
val yStd = math.sqrt(ySummarizer.variance(0))
+ if ($(solver) != "l-bfgs" && ($(solver) == "normal" ||
+ ($(elasticNetParam) == 0.0 && numFeatures <= 4096))) {
+ // In case of feature size is small, WeightedLeastSquares can train more efficiently
+ // because it requires one pass through to the data. (SPARK-10668)
+ val instances: RDD[WeightedLeastSquares.Instance] = dataset.select(
+ col($(labelCol)), w, col($(featuresCol))).map {
+ case Row(label: Double, weight: Double, features: Vector) =>
+ WeightedLeastSquares.Instance(weight, features, label)
--- End diff --
indent
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40746290
--- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -169,6 +181,33 @@ class LinearRegression(override val uid: String)
val yMean = ySummarizer.mean(0)
val yStd = math.sqrt(ySummarizer.variance(0))
--- End diff --
That's okay.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147726522
[Test build #43643 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43643/console) for PR 8884 at commit [`8583f4a`](https://github.com/apache/spark/commit/8583f4acc2424ad75e8e66e8c4f6d2f6e8d86ab2).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41810212
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,559 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
--- End diff --
The `{` in ` solver => {` is not needed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147714575
[Test build #43643 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43643/consoleFull) for PR 8884 at commit [`8583f4a`](https://github.com/apache/spark/commit/8583f4acc2424ad75e8e66e8c4f6d2f6e8d86ab2).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147281110
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147928769
[Test build #43700 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43700/console) for PR 8884 at commit [`23a8ab1`](https://github.com/apache/spark/commit/23a8ab148f196ca2716eca890d68e5d3d21c85c3).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* ` def getPath = path.getOrElse(sys.error("Constructors must start at a class type"))`
* `case class ExprId(id: Long, jvmId: UUID)`
* `case class WrapOption(optionType: DataType, child: Expression)`
* `class GenericArrayData(val array: Array[Any]) extends ArrayData `
* `trait QueryExecutionListener `
* `class ExecutionListenerManager extends Logging `
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-148769776
@Lewuathe
They should be the same, but there is no test proving it.
Since in
```scala
test("linear regression with weighted samples"){
```
the normal one is excluded due to the L1 issue, can you add another test without L1 to prove that `normal` and `l-bfgs` obtain the same solution? Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r42401838
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,603 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with only L1 regularization case.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+
--- End diff --
also, move `var model1` in here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by Lewuathe <gi...@git.apache.org>.
Github user Lewuathe commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-149143395
@dbtsai Oh. Thanks I updated.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142758269
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42929/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147186458
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147182511
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43541/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40518663
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -255,29 +264,32 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
as.numeric.data.V2. 6.232193
as.numeric.data.V3. 4.764229
*/
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(6.232193, 4.764229)
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(6.232193, 4.764229)
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ })
}
test("linear regression with intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
+ Seq("auto", "l-bfgs", "normal").foreach(solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
--- End diff --
ditto
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41808061
--- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -130,9 +131,54 @@ class LinearRegression(override val uid: String)
def setWeightCol(value: String): this.type = set(weightCol, value)
setDefault(weightCol -> "")
+ /**
+ * Set the solver algorithm used for optimization.
+ * In case of linear regression, this can be "l-bfgs", "normal" and "auto".
+ * The default value is "auto" which means that the solver algorithm is
+ * selected automatically.
+ * @group setParam
+ */
+ def setSolver(value: String): this.type = set(solver, value)
+ setDefault(solver -> "auto")
+
override protected def train(dataset: DataFrame): LinearRegressionModel = {
+ // Extract the number of features before deciding optimization solver.
+ val numFeatures = dataset.select(col($(featuresCol))).limit(1).map {
+ case Row(features: Vector) =>
+ features.size
--- End diff --
move `features.size` to the line of `case Row...`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by Lewuathe <gi...@git.apache.org>.
Github user Lewuathe commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-148679109
> Please add a test for this. If they are in fact returning different solution, then throw exception saying that weighted LiR doesn't support normal mode
@dbtsai Should it be same solution even if the solver mode is exclusive?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-143178038
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-144384761
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-144382988
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147912959
[Test build #43699 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43699/consoleFull) for PR 8884 at commit [`ea5f195`](https://github.com/apache/spark/commit/ea5f195f8b3fce7dac5b57750ef44bb59c9a0eb3).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147919705
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43699/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41808590
--- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -130,9 +131,54 @@ class LinearRegression(override val uid: String)
def setWeightCol(value: String): this.type = set(weightCol, value)
setDefault(weightCol -> "")
+ /**
+ * Set the solver algorithm used for optimization.
+ * In case of linear regression, this can be "l-bfgs", "normal" and "auto".
+ * The default value is "auto" which means that the solver algorithm is
+ * selected automatically.
+ * @group setParam
+ */
+ def setSolver(value: String): this.type = set(solver, value)
+ setDefault(solver -> "auto")
+
override protected def train(dataset: DataFrame): LinearRegressionModel = {
+ // Extract the number of features before deciding optimization solver.
+ val numFeatures = dataset.select(col($(featuresCol))).limit(1).map {
+ case Row(features: Vector) =>
+ features.size
+ }.toArray()(0)
// Extract columns from data. If dataset is persisted, do not persist instances.
--- End diff --
Remove `// Extract columns from data. If dataset is persisted, do not persist instances.`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r42328431
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +96,641 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with only L1 regularization case.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ break
+ } else {
--- End diff --
break will break the entire loop, so it depends on the ordering of Seq, and it only works for the last element in the Seq. This will easily cause some bugs.
See,
```scala
scala> :paste
// Entering paste mode (ctrl-D to finish)
import scala.util.control.Breaks.break
import scala.util.control.Breaks.breakable
breakable {
Seq("auto", "l-bfgs", "normal").foreach { solver =>
println(s"starting: $solver")
if (solver == "l-bfgs") {
break
}
println(s"ending: $solver")
}
}
// Exiting paste mode, now interpreting.
starting: auto
ending: auto
starting: l-bfgs
```
I suggest that move the whole test code into else statement,
```scala
} else {
var model1 = trainer1.fit(dataset)
val model2 = trainer2.fit(dataset)
val interceptR1 = 6.24300
val weightsR1 = Vectors.dense(4.024821, 6.679841)
assert(model1.intercept ~== interceptR1 relTol 1E-3)
assert(model1.weights ~= weightsR1 relTol 1E-3)
...
...
}
```
Also, fix the indentation in the comment.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147928870
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-148321601
I'll be able to merge this PR once the following two issues are addressed,
1) Have a test to verify that `L1` with normal will throw `IllegalArugmentExcpetion`. I prefer have the test inside `Seq("auto", "l-bfgs") since I'll work on solving L1 with `normal` later, so what we need to do it just remove the `Intercept`. Also, we can precise know what part is not supported if we have the tests together.
2) Currently, we allow weighted LiR in `normal` mode, but we don't have a test verifying they get the same solution. Please add a test for this. If they are in fact returning different solution, then throw exception saying that weighted LiR doesn't support `normal` mode, and in `auto` mode with weighted LiR, we should do `l-bfgs`. Then we address the inconsistent issue in a separated PR.
Thanks. Once those two issues are solved, I'll merge it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41811141
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,559 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }}
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.24300
+ as.numeric.data.V2. 4.024821
+ as.numeric.data.V3. 6.679841
+ */
+ val interceptR1 = 6.24300
+ val weightsR1 = Vectors.dense(4.024821, 6.679841)
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.416948
+ as.numeric.data.V2. 3.893869
+ as.numeric.data.V3. 6.724286
+ */
+ val interceptR2 = 6.416948
+ val weightsR2 = Vectors.dense(3.893869, 6.724286)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.299752
- as.numeric.data.V3. 4.772913
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(6.299752, 4.772913)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.232193
- as.numeric.data.V3. 4.764229
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(6.232193, 4.764229)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.299752
+ as.numeric.data.V3. 4.772913
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(6.299752, 4.772913)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.232193
+ as.numeric.data.V3. 4.764229
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(6.232193, 4.764229)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression with intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.269376
- as.numeric.data.V2. 3.736216
- as.numeric.data.V3. 5.712356)
- */
- val interceptR1 = 5.269376
- val weightsR1 = Vectors.dense(3.736216, 5.712356)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.791109
- as.numeric.data.V2. 3.435466
- as.numeric.data.V3. 5.910406
- */
- val interceptR2 = 5.791109
- val weightsR2 = Vectors.dense(3.435466, 5.910406)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.269376
+ as.numeric.data.V2. 3.736216
+ as.numeric.data.V3. 5.712356)
+ */
+ val interceptR1 = 5.269376
+ val weightsR1 = Vectors.dense(3.736216, 5.712356)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.791109
+ as.numeric.data.V2. 3.435466
+ as.numeric.data.V3. 5.910406
+ */
+ val interceptR2 = 5.791109
+ val weightsR2 = Vectors.dense(3.435466, 5.910406)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.522875
- as.numeric.data.V3. 4.214502
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.522875, 4.214502)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.263704
- as.numeric.data.V3. 4.187419
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.263704, 4.187419)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.522875
+ as.numeric.data.V3. 4.214502
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.522875, 4.214502)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.263704
+ as.numeric.data.V3. 4.187419
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.263704, 4.187419)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression with intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.324108
- as.numeric.data.V2. 3.168435
- as.numeric.data.V3. 5.200403
- */
- val interceptR1 = 5.696056
- val weightsR1 = Vectors.dense(3.670489, 6.001122)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.114723
- as.numeric.data.V2. 3.409937
- as.numeric.data.V3. 6.146531
- */
- val interceptR2 = 6.114723
- val weightsR2 = Vectors.dense(3.409937, 6.146531)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.324108
+ as.numeric.data.V2. 3.168435
+ as.numeric.data.V3. 5.200403
+ */
+ val interceptR1 = 5.696056
+ val weightsR1 = Vectors.dense(3.670489, 6.001122)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.114723
+ as.numeric.data.V2. 3.409937
+ as.numeric.data.V3. 6.146531
+ */
+ val interceptR2 = 6.114723
+ val weightsR2 = Vectors.dense(3.409937, 6.146531)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.dataM.V2. 5.673348
- as.numeric.dataM.V3. 4.322251
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.673348, 4.322251)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.477988
- as.numeric.data.V3. 4.297622
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.477988, 4.297622)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.dataM.V2. 5.673348
+ as.numeric.dataM.V3. 4.322251
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.673348, 4.322251)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.477988
+ as.numeric.data.V3. 4.297622
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.477988, 4.297622)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
--- End diff --
ditto
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41810986
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,559 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }}
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
--- End diff --
```scala
var model1: LinearRegressionModel
var model2: LinearRegressionModel
if (solver == "normal") {
intercept[IllegalArgumentException] {
model1 = trainer1.fit(dataset)
}
intercept[IllegalArgumentException] {
model2 = trainer2.fit(dataset)
}
} else {
model1 = trainer1.fit(dataset)
model2 = trainer2.fit(dataset)
}
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142599791
[Test build #42907 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42907/consoleFull) for PR 8884 at commit [`5ca2a39`](https://github.com/apache/spark/commit/5ca2a3980a5a3820b125ab84481382edcf386c73).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-144385288
[Test build #43131 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43131/consoleFull) for PR 8884 at commit [`746cc5f`](https://github.com/apache/spark/commit/746cc5fdb16ff4ec265bd68efed9277c7b49d9c4).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-144383036
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147919701
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147911717
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-149131663
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43920/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by yanboliang <gi...@git.apache.org>.
Github user yanboliang commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40528438
--- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -169,6 +181,33 @@ class LinearRegression(override val uid: String)
val yMean = ySummarizer.mean(0)
val yStd = math.sqrt(ySummarizer.variance(0))
+ if ($(solver) != "l-bfgs" && ($(solver) == "normal" ||
+ ($(elasticNetParam) == 0.0 && numFeatures <= 4096))) {
--- End diff --
Your code is right, but I think the following style is more comprehensible
```scala
if ($(solver) == "normal" || ($(solver) == "auto" && $(elasticNetParam) == 0.0 && numFeatures <= 4096)
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40518589
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -166,22 +171,25 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
as.numeric.data3.V2. 4.70011
as.numeric.data3.V3. 7.19943
*/
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ })
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
--- End diff --
Can you add another test that with L1, using `normal` as solver will throw exception?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142580372
[Test build #42905 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42905/consoleFull) for PR 8884 at commit [`570a558`](https://github.com/apache/spark/commit/570a5588858d690c240d5baa979fa2510f74b705).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142578572
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41810342
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,559 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
--- End diff --
Remove the extra `}`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41807118
--- Diff: mllib/src/main/scala/org/apache/spark/ml/param/shared/SharedParamsCodeGen.scala ---
@@ -73,7 +73,9 @@ private[shared] object SharedParamsCodeGen {
ParamDesc[Double]("tol", "the convergence tolerance for iterative algorithms"),
ParamDesc[Double]("stepSize", "Step size to be used for each iteration of optimization."),
ParamDesc[String]("weightCol", "weight column name. If this is not set or empty, we treat " +
- "all instance weights as 1.0."))
+ "all instance weights as 1.0."),
+ ParamDesc[String]("solver", "the solver algorithm for optimization. If this is not set or " +
+ "empty, default value is 'auto'."))
--- End diff --
```scala
ParamDesc[String]("solver", "the solver algorithm for optimization. If this is not set or " +
"empty, default value is 'auto'."), Some("\"auto\""))
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by Lewuathe <gi...@git.apache.org>.
Github user Lewuathe commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-149009216
@dbtsai Thank you so much again. Could check this please?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142790549
[Test build #42950 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42950/console) for PR 8884 at commit [`fa01cae`](https://github.com/apache/spark/commit/fa01cae9e7358b932cdab7d7b3d07fa5a28c44fb).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142749155
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-143134214
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43009/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-144384779
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-149007581
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41899983
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,559 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }}
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.24300
+ as.numeric.data.V2. 4.024821
+ as.numeric.data.V3. 6.679841
+ */
+ val interceptR1 = 6.24300
+ val weightsR1 = Vectors.dense(4.024821, 6.679841)
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.416948
+ as.numeric.data.V2. 3.893869
+ as.numeric.data.V3. 6.724286
+ */
+ val interceptR2 = 6.416948
+ val weightsR2 = Vectors.dense(3.893869, 6.724286)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.299752
- as.numeric.data.V3. 4.772913
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(6.299752, 4.772913)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.232193
- as.numeric.data.V3. 4.764229
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(6.232193, 4.764229)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.299752
+ as.numeric.data.V3. 4.772913
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(6.299752, 4.772913)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.232193
+ as.numeric.data.V3. 4.764229
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(6.232193, 4.764229)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression with intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.269376
- as.numeric.data.V2. 3.736216
- as.numeric.data.V3. 5.712356)
- */
- val interceptR1 = 5.269376
- val weightsR1 = Vectors.dense(3.736216, 5.712356)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.791109
- as.numeric.data.V2. 3.435466
- as.numeric.data.V3. 5.910406
- */
- val interceptR2 = 5.791109
- val weightsR2 = Vectors.dense(3.435466, 5.910406)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.269376
+ as.numeric.data.V2. 3.736216
+ as.numeric.data.V3. 5.712356)
+ */
+ val interceptR1 = 5.269376
+ val weightsR1 = Vectors.dense(3.736216, 5.712356)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.791109
+ as.numeric.data.V2. 3.435466
+ as.numeric.data.V3. 5.910406
+ */
+ val interceptR2 = 5.791109
+ val weightsR2 = Vectors.dense(3.435466, 5.910406)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.522875
- as.numeric.data.V3. 4.214502
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.522875, 4.214502)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.263704
- as.numeric.data.V3. 4.187419
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.263704, 4.187419)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.522875
+ as.numeric.data.V3. 4.214502
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.522875, 4.214502)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.263704
+ as.numeric.data.V3. 4.187419
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.263704, 4.187419)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression with intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.324108
- as.numeric.data.V2. 3.168435
- as.numeric.data.V3. 5.200403
- */
- val interceptR1 = 5.696056
- val weightsR1 = Vectors.dense(3.670489, 6.001122)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.114723
- as.numeric.data.V2. 3.409937
- as.numeric.data.V3. 6.146531
- */
- val interceptR2 = 6.114723
- val weightsR2 = Vectors.dense(3.409937, 6.146531)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.324108
+ as.numeric.data.V2. 3.168435
+ as.numeric.data.V3. 5.200403
+ */
+ val interceptR1 = 5.696056
+ val weightsR1 = Vectors.dense(3.670489, 6.001122)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.114723
+ as.numeric.data.V2. 3.409937
+ as.numeric.data.V3. 6.146531
+ */
+ val interceptR2 = 6.114723
+ val weightsR2 = Vectors.dense(3.409937, 6.146531)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.dataM.V2. 5.673348
- as.numeric.dataM.V3. 4.322251
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.673348, 4.322251)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.477988
- as.numeric.data.V3. 4.297622
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.477988, 4.297622)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.dataM.V2. 5.673348
+ as.numeric.dataM.V3. 4.322251
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.673348, 4.322251)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.477988
+ as.numeric.data.V3. 4.297622
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.477988, 4.297622)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression model training summary") {
- val trainer = new LinearRegression
- val model = trainer.fit(dataset)
- val trainerNoPredictionCol = trainer.setPredictionCol("")
- val modelNoPredictionCol = trainerNoPredictionCol.fit(dataset)
-
-
- // Training results for the model should be available
- assert(model.hasSummary)
- assert(modelNoPredictionCol.hasSummary)
-
- // Schema should be a superset of the input dataset
- assert((dataset.schema.fieldNames.toSet + "prediction").subsetOf(
- model.summary.predictions.schema.fieldNames.toSet))
- // Validate that we re-insert a prediction column for evaluation
- val modelNoPredictionColFieldNames = modelNoPredictionCol.summary.predictions.schema.fieldNames
- assert((dataset.schema.fieldNames.toSet).subsetOf(
- modelNoPredictionColFieldNames.toSet))
- assert(modelNoPredictionColFieldNames.exists(s => s.startsWith("prediction_")))
-
- // Residuals in [[LinearRegressionResults]] should equal those manually computed
- val expectedResiduals = dataset.select("features", "label")
- .map { case Row(features: DenseVector, label: Double) =>
- val prediction =
- features(0) * model.weights(0) + features(1) * model.weights(1) + model.intercept
- label - prediction
- }
- .zip(model.summary.residuals.map(_.getDouble(0)))
- .collect()
- .foreach { case (manualResidual: Double, resultResidual: Double) =>
- assert(manualResidual ~== resultResidual relTol 1E-5)
- }
+ Seq("l-bfgs").foreach { solver => {
+ val trainer = new LinearRegression().setSolver(solver)
+ val model = trainer.fit(dataset)
+ val trainerNoPredictionCol = trainer.setPredictionCol("")
+ val modelNoPredictionCol = trainerNoPredictionCol.fit(dataset)
+
+
+ // Training results for the model should be available
+ assert(model.hasSummary)
+ assert(modelNoPredictionCol.hasSummary)
+
+ // Schema should be a superset of the input dataset
+ assert((dataset.schema.fieldNames.toSet + "prediction").subsetOf(
+ model.summary.predictions.schema.fieldNames.toSet))
+ // Validate that we re-insert a prediction column for evaluation
+ val modelNoPredictionColFieldNames
+ = modelNoPredictionCol.summary.predictions.schema.fieldNames
+ assert((dataset.schema.fieldNames.toSet).subsetOf(
+ modelNoPredictionColFieldNames.toSet))
+ assert(modelNoPredictionColFieldNames.exists(s => s.startsWith("prediction_")))
+
+ // Residuals in [[LinearRegressionResults]] should equal those manually computed
+ val expectedResiduals = dataset.select("features", "label")
+ .map { case Row(features: DenseVector, label: Double) =>
+ val prediction =
+ features(0) * model.weights(0) + features(1) * model.weights(1) + model.intercept
+ label - prediction
+ }
+ .zip(model.summary.residuals.map(_.getDouble(0)))
+ .collect()
+ .foreach { case (manualResidual: Double, resultResidual: Double) =>
+ assert(manualResidual ~== resultResidual relTol 1E-5)
+ }
- /*
- Use the following R code to generate model training results.
-
- predictions <- predict(fit, newx=features)
- residuals <- label - predictions
- > mean(residuals^2) # MSE
- [1] 0.009720325
- > mean(abs(residuals)) # MAD
- [1] 0.07863206
- > cor(predictions, label)^2# r^2
- [,1]
- s0 0.9998749
- */
- assert(model.summary.meanSquaredError ~== 0.00972035 relTol 1E-5)
- assert(model.summary.meanAbsoluteError ~== 0.07863206 relTol 1E-5)
- assert(model.summary.r2 ~== 0.9998749 relTol 1E-5)
-
- // Objective function should be monotonically decreasing for linear regression
- assert(
- model.summary
- .objectiveHistory
- .sliding(2)
- .forall(x => x(0) >= x(1)))
+ /*
+ Use the following R code to generate model training results.
+
+ predictions <- predict(fit, newx=features)
+ residuals <- label - predictions
+ > mean(residuals^2) # MSE
+ [1] 0.009720325
+ > mean(abs(residuals)) # MAD
+ [1] 0.07863206
+ > cor(predictions, label)^2# r^2
+ [,1]
+ s0 0.9998749
+ */
+ assert(model.summary.meanSquaredError ~== 0.00972035 relTol 1E-5)
+ assert(model.summary.meanAbsoluteError ~== 0.07863206 relTol 1E-5)
+ assert(model.summary.r2 ~== 0.9998749 relTol 1E-5)
+
+ // Normal solver uses "WeightedLeastSquares". This algorithm does not generate
+ // objective history because it does not run through iterations.
+ if (solver == "l-bfgs") {
+ // Objective function should be monotonically decreasing for linear regression
+ assert(
+ model.summary
+ .objectiveHistory
+ .sliding(2)
+ .forall(x => x(0) >= x(1)))
+ }
+ }}
}
test("linear regression model testset evaluation summary") {
- val trainer = new LinearRegression
- val model = trainer.fit(dataset)
-
- // Evaluating on training dataset should yield results summary equal to training summary
- val testSummary = model.evaluate(dataset)
- assert(model.summary.meanSquaredError ~== testSummary.meanSquaredError relTol 1E-5)
- assert(model.summary.r2 ~== testSummary.r2 relTol 1E-5)
- model.summary.residuals.select("residuals").collect()
- .zip(testSummary.residuals.select("residuals").collect())
- .forall { case (Row(r1: Double), Row(r2: Double)) => r1 ~== r2 relTol 1E-5 }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer = new LinearRegression().setSolver(solver)
+ val model = trainer.fit(dataset)
+
+ // Evaluating on training dataset should yield results summary equal to training summary
+ val testSummary = model.evaluate(dataset)
+ assert(model.summary.meanSquaredError ~== testSummary.meanSquaredError relTol 1E-5)
+ assert(model.summary.r2 ~== testSummary.r2 relTol 1E-5)
+ model.summary.residuals.select("residuals").collect()
+ .zip(testSummary.residuals.select("residuals").collect())
+ .forall { case (Row(r1: Double), Row(r2: Double)) => r1 ~== r2 relTol 1E-5 }
+ }}
}
test("linear regression with weighted samples"){
- val (data, weightedData) = {
- val activeData = LinearDataGenerator.generateLinearInput(
- 6.3, Array(4.7, 7.2), Array(0.9, -1.3), Array(0.7, 1.2), 500, 1, 0.1)
-
- val rnd = new Random(8392)
- val signedData = activeData.map { case p: LabeledPoint =>
- (rnd.nextGaussian() > 0.0, p)
- }
-
- val data1 = signedData.flatMap {
- case (true, p) => Iterator(p, p)
- case (false, p) => Iterator(p)
- }
-
- val weightedSignedData = signedData.flatMap {
- case (true, LabeledPoint(label, features)) =>
- Iterator(
- Instance(label, weight = 1.2, features),
- Instance(label, weight = 0.8, features)
- )
- case (false, LabeledPoint(label, features)) =>
- Iterator(
- Instance(label, weight = 0.3, features),
- Instance(label, weight = 0.1, features),
- Instance(label, weight = 0.6, features)
- )
+ Seq("auto", "l-bfgs").foreach { solver => {
--- End diff --
Can you add a simple test testing that it will match for only L2? Or just make this test to L2 should be good for me.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41806848
--- Diff: R/pkg/R/mllib.R ---
@@ -45,12 +45,13 @@ setClass("PipelineModel", representation(model = "jobj"))
#' summary(model)
#'}
setMethod("glm", signature(formula = "formula", family = "ANY", data = "DataFrame"),
- function(formula, family = c("gaussian", "binomial"), data, lambda = 0, alpha = 0) {
- family <- match.arg(family)
- model <- callJStatic("org.apache.spark.ml.api.r.SparkRWrappers",
+ function(formula, family = c("gaussian", "binomial"), data, lambda = 0, alpha = 0,
+ solver = "auto") {
+ family <- match.arg(family)
+ model <- callJStatic("org.apache.spark.ml.api.r.SparkRWrappers",
"fitRModelFormula", deparse(formula), data@sdf, family, lambda,
--- End diff --
ditto
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40519125
--- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -169,6 +181,33 @@ class LinearRegression(override val uid: String)
val yMean = ySummarizer.mean(0)
val yStd = math.sqrt(ySummarizer.variance(0))
+ if ($(solver) != "l-bfgs" && ($(solver) == "normal" ||
+ ($(elasticNetParam) == 0.0 && numFeatures <= 4096))) {
+ // In case of feature size is small, WeightedLeastSquares can train more efficiently
+ // because it requires one pass through to the data. (SPARK-10668)
+ val instances: RDD[WeightedLeastSquares.Instance] = dataset.select(
+ col($(labelCol)), w, col($(featuresCol))).map {
+ case Row(label: Double, weight: Double, features: Vector) =>
--- End diff --
indentation
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41806833
--- Diff: R/pkg/R/mllib.R ---
@@ -45,12 +45,13 @@ setClass("PipelineModel", representation(model = "jobj"))
#' summary(model)
#'}
setMethod("glm", signature(formula = "formula", family = "ANY", data = "DataFrame"),
- function(formula, family = c("gaussian", "binomial"), data, lambda = 0, alpha = 0) {
- family <- match.arg(family)
- model <- callJStatic("org.apache.spark.ml.api.r.SparkRWrappers",
+ function(formula, family = c("gaussian", "binomial"), data, lambda = 0, alpha = 0,
+ solver = "auto") {
+ family <- match.arg(family)
--- End diff --
remove two spaces.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-143121010
[Test build #43007 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43007/consoleFull) for PR 8884 at commit [`78677dd`](https://github.com/apache/spark/commit/78677dda2e84a02eaa415347254ca446b5238a43).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40518453
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -92,13 +93,14 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
+ Seq("auto", "l-bfgs", "normal").foreach(solver => {
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
--- End diff --
Update the indentation of the R example.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r42327468
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +96,641 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with only L1 regularization case.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ break
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+ }
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.24300
+ as.numeric.data.V2. 4.024821
+ as.numeric.data.V3. 6.679841
+ */
+ val interceptR1 = 6.24300
+ val weightsR1 = Vectors.dense(4.024821, 6.679841)
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.416948
+ as.numeric.data.V2. 3.893869
+ as.numeric.data.V3. 6.724286
+ */
+ val interceptR2 = 6.416948
+ val weightsR2 = Vectors.dense(3.893869, 6.724286)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression without intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.299752
- as.numeric.data.V3. 4.772913
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(6.299752, 4.772913)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.232193
- as.numeric.data.V3. 4.764229
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(6.232193, 4.764229)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with only L1 regularization case.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ break
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+ }
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.299752
+ as.numeric.data.V3. 4.772913
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(6.299752, 4.772913)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.232193
+ as.numeric.data.V3. 4.764229
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(6.232193, 4.764229)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression with intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.269376
- as.numeric.data.V2. 3.736216
- as.numeric.data.V3. 5.712356)
- */
- val interceptR1 = 5.269376
- val weightsR1 = Vectors.dense(3.736216, 5.712356)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.791109
- as.numeric.data.V2. 3.435466
- as.numeric.data.V3. 5.910406
- */
- val interceptR2 = 5.791109
- val weightsR2 = Vectors.dense(3.435466, 5.910406)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.269376
+ as.numeric.data.V2. 3.736216
+ as.numeric.data.V3. 5.712356)
+ */
+ val interceptR1 = 5.269376
+ val weightsR1 = Vectors.dense(3.736216, 5.712356)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.791109
+ as.numeric.data.V2. 3.435466
+ as.numeric.data.V3. 5.910406
+ */
+ val interceptR2 = 5.791109
+ val weightsR2 = Vectors.dense(3.435466, 5.910406)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression without intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.522875
- as.numeric.data.V3. 4.214502
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.522875, 4.214502)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.263704
- as.numeric.data.V3. 4.187419
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.263704, 4.187419)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.522875
+ as.numeric.data.V3. 4.214502
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.522875, 4.214502)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.263704
+ as.numeric.data.V3. 4.187419
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.263704, 4.187419)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression with intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.324108
- as.numeric.data.V2. 3.168435
- as.numeric.data.V3. 5.200403
- */
- val interceptR1 = 5.696056
- val weightsR1 = Vectors.dense(3.670489, 6.001122)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.114723
- as.numeric.data.V2. 3.409937
- as.numeric.data.V3. 6.146531
- */
- val interceptR2 = 6.114723
- val weightsR2 = Vectors.dense(3.409937, 6.146531)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setStandardization(false).setSolver(solver)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with non-zero elasticnet parameter.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ break
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+ }
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.324108
+ as.numeric.data.V2. 3.168435
+ as.numeric.data.V3. 5.200403
+ */
+ val interceptR1 = 5.696056
+ val weightsR1 = Vectors.dense(3.670489, 6.001122)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.114723
+ as.numeric.data.V2. 3.409937
+ as.numeric.data.V3. 6.146531
+ */
+ val interceptR2 = 6.114723
+ val weightsR2 = Vectors.dense(3.409937, 6.146531)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression without intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.dataM.V2. 5.673348
- as.numeric.dataM.V3. 4.322251
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.673348, 4.322251)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.477988
- as.numeric.data.V3. 4.297622
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.477988, 4.297622)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with non-zero elasticnet parameter.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ break
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+ }
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.dataM.V2. 5.673348
+ as.numeric.dataM.V3. 4.322251
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.673348, 4.322251)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.477988
+ as.numeric.data.V3. 4.297622
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.477988, 4.297622)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression model training summary") {
- val trainer = new LinearRegression
- val model = trainer.fit(dataset)
- val trainerNoPredictionCol = trainer.setPredictionCol("")
- val modelNoPredictionCol = trainerNoPredictionCol.fit(dataset)
-
-
- // Training results for the model should be available
- assert(model.hasSummary)
- assert(modelNoPredictionCol.hasSummary)
-
- // Schema should be a superset of the input dataset
- assert((dataset.schema.fieldNames.toSet + "prediction").subsetOf(
- model.summary.predictions.schema.fieldNames.toSet))
- // Validate that we re-insert a prediction column for evaluation
- val modelNoPredictionColFieldNames = modelNoPredictionCol.summary.predictions.schema.fieldNames
- assert((dataset.schema.fieldNames.toSet).subsetOf(
- modelNoPredictionColFieldNames.toSet))
- assert(modelNoPredictionColFieldNames.exists(s => s.startsWith("prediction_")))
-
- // Residuals in [[LinearRegressionResults]] should equal those manually computed
- val expectedResiduals = dataset.select("features", "label")
- .map { case Row(features: DenseVector, label: Double) =>
- val prediction =
- features(0) * model.weights(0) + features(1) * model.weights(1) + model.intercept
- label - prediction
- }
- .zip(model.summary.residuals.map(_.getDouble(0)))
- .collect()
- .foreach { case (manualResidual: Double, resultResidual: Double) =>
- assert(manualResidual ~== resultResidual relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer = new LinearRegression().setSolver(solver)
+ val model = trainer.fit(dataset)
+ val trainerNoPredictionCol = trainer.setPredictionCol("")
+ val modelNoPredictionCol = trainerNoPredictionCol.fit(dataset)
+
+
+ // Training results for the model should be available
+ assert(model.hasSummary)
+ assert(modelNoPredictionCol.hasSummary)
+
+ // Schema should be a superset of the input dataset
+ assert((dataset.schema.fieldNames.toSet + "prediction").subsetOf(
+ model.summary.predictions.schema.fieldNames.toSet))
+ // Validate that we re-insert a prediction column for evaluation
+ val modelNoPredictionColFieldNames
+ = modelNoPredictionCol.summary.predictions.schema.fieldNames
+ assert((dataset.schema.fieldNames.toSet).subsetOf(
+ modelNoPredictionColFieldNames.toSet))
+ assert(modelNoPredictionColFieldNames.exists(s => s.startsWith("prediction_")))
+
+ // Residuals in [[LinearRegressionResults]] should equal those manually computed
+ val expectedResiduals = dataset.select("features", "label")
+ .map { case Row(features: DenseVector, label: Double) =>
+ val prediction =
+ features(0) * model.weights(0) + features(1) * model.weights(1) + model.intercept
+ label - prediction
+ }
+ .zip(model.summary.residuals.map(_.getDouble(0)))
+ .collect()
+ .foreach { case (manualResidual: Double, resultResidual: Double) =>
+ assert(manualResidual ~== resultResidual relTol 1E-5)
+ }
- /*
- Use the following R code to generate model training results.
-
- predictions <- predict(fit, newx=features)
- residuals <- label - predictions
- > mean(residuals^2) # MSE
- [1] 0.009720325
- > mean(abs(residuals)) # MAD
- [1] 0.07863206
- > cor(predictions, label)^2# r^2
- [,1]
- s0 0.9998749
- */
- assert(model.summary.meanSquaredError ~== 0.00972035 relTol 1E-5)
- assert(model.summary.meanAbsoluteError ~== 0.07863206 relTol 1E-5)
- assert(model.summary.r2 ~== 0.9998749 relTol 1E-5)
-
- // Objective function should be monotonically decreasing for linear regression
- assert(
- model.summary
- .objectiveHistory
- .sliding(2)
- .forall(x => x(0) >= x(1)))
+ /*
+ Use the following R code to generate model training results.
+
+ predictions <- predict(fit, newx=features)
+ residuals <- label - predictions
+ > mean(residuals^2) # MSE
+ [1] 0.009720325
+ > mean(abs(residuals)) # MAD
+ [1] 0.07863206
+ > cor(predictions, label)^2# r^2
+ [,1]
+ s0 0.9998749
+ */
+ assert(model.summary.meanSquaredError ~== 0.00972035 relTol 1E-5)
+ assert(model.summary.meanAbsoluteError ~== 0.07863206 relTol 1E-5)
+ assert(model.summary.r2 ~== 0.9998749 relTol 1E-5)
+
+ // Normal solver uses "WeightedLeastSquares". This algorithm does not generate
+ // objective history because it does not run through iterations.
+ if (solver == "l-bfgs") {
+ // Objective function should be monotonically decreasing for linear regression
+ assert(
+ model.summary
+ .objectiveHistory
+ .sliding(2)
+ .forall(x => x(0) >= x(1)))
+ }
+ }
}
test("linear regression model testset evaluation summary") {
- val trainer = new LinearRegression
- val model = trainer.fit(dataset)
-
- // Evaluating on training dataset should yield results summary equal to training summary
- val testSummary = model.evaluate(dataset)
- assert(model.summary.meanSquaredError ~== testSummary.meanSquaredError relTol 1E-5)
- assert(model.summary.r2 ~== testSummary.r2 relTol 1E-5)
- model.summary.residuals.select("residuals").collect()
- .zip(testSummary.residuals.select("residuals").collect())
- .forall { case (Row(r1: Double), Row(r2: Double)) => r1 ~== r2 relTol 1E-5 }
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer = new LinearRegression().setSolver(solver)
+ val model = trainer.fit(dataset)
+
+ // Evaluating on training dataset should yield results summary equal to training summary
+ val testSummary = model.evaluate(dataset)
+ assert(model.summary.meanSquaredError ~== testSummary.meanSquaredError relTol 1E-5)
+ assert(model.summary.r2 ~== testSummary.r2 relTol 1E-5)
+ model.summary.residuals.select("residuals").collect()
+ .zip(testSummary.residuals.select("residuals").collect())
+ .forall { case (Row(r1: Double), Row(r2: Double)) => r1 ~== r2 relTol 1E-5 }
+ }
}
test("linear regression with weighted samples"){
- val (data, weightedData) = {
- val activeData = LinearDataGenerator.generateLinearInput(
- 6.3, Array(4.7, 7.2), Array(0.9, -1.3), Array(0.7, 1.2), 500, 1, 0.1)
-
- val rnd = new Random(8392)
- val signedData = activeData.map { case p: LabeledPoint =>
- (rnd.nextGaussian() > 0.0, p)
- }
-
- val data1 = signedData.flatMap {
- case (true, p) => Iterator(p, p)
- case (false, p) => Iterator(p)
- }
-
- val weightedSignedData = signedData.flatMap {
- case (true, LabeledPoint(label, features)) =>
- Iterator(
- Instance(label, weight = 1.2, features),
- Instance(label, weight = 0.8, features)
- )
- case (false, LabeledPoint(label, features)) =>
- Iterator(
- Instance(label, weight = 0.3, features),
- Instance(label, weight = 0.1, features),
- Instance(label, weight = 0.6, features)
- )
- }
-
- val noiseData = LinearDataGenerator.generateLinearInput(
- 2, Array(1, 3), Array(0.9, -1.3), Array(0.7, 1.2), 500, 1, 0.1)
- val weightedNoiseData = noiseData.map {
- case LabeledPoint(label, features) => Instance(label, weight = 0, features)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val (data, weightedData) = {
+ val activeData = LinearDataGenerator.generateLinearInput(
+ 6.3, Array(4.7, 7.2), Array(0.9, -1.3), Array(0.7, 1.2), 500, 1, 0.1)
+
+ val rnd = new Random(8392)
+ val signedData = activeData.map { case p: LabeledPoint =>
+ (rnd.nextGaussian() > 0.0, p)
+ }
+
+ val data1 = signedData.flatMap {
+ case (true, p) => Iterator(p, p)
+ case (false, p) => Iterator(p)
+ }
+
+ val weightedSignedData = signedData.flatMap {
+ case (true, LabeledPoint(label, features)) =>
+ Iterator(
+ Instance(label, weight = 1.2, features),
+ Instance(label, weight = 0.8, features)
+ )
+ case (false, LabeledPoint(label, features)) =>
+ Iterator(
+ Instance(label, weight = 0.3, features),
+ Instance(label, weight = 0.1, features),
+ Instance(label, weight = 0.6, features)
+ )
+ }
+
+ val noiseData = LinearDataGenerator.generateLinearInput(
+ 2, Array(1, 3), Array(0.9, -1.3), Array(0.7, 1.2), 500, 1, 0.1)
+ val weightedNoiseData = noiseData.map {
+ case LabeledPoint(label, features) => Instance(label, weight = 0, features)
+ }
+ val data2 = weightedSignedData ++ weightedNoiseData
+
+ (sqlContext.createDataFrame(sc.parallelize(data1, 4)),
+ sqlContext.createDataFrame(sc.parallelize(data2, 4)))
+ }
+
+ val trainer1a = (new LinearRegression).setFitIntercept(true)
+ .setElasticNetParam(0.38).setRegParam(0.21).setStandardization(true).setSolver(solver)
+ val trainer1b = (new LinearRegression).setFitIntercept(true).setWeightCol("weight")
+ .setElasticNetParam(0.38).setRegParam(0.21).setStandardization(true).setSolver(solver)
+
+ var model1a0: LinearRegressionModel = null
+ var model1a1: LinearRegressionModel = null
+ var model1b: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with non-zero elasticnet parameter.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1a.fit(dataset)
+ trainer1a.fit(weightedData)
+ trainer1b.fit(weightedData)
+ }
+ break
+ } else {
+ model1a0 = trainer1a.fit(data)
+ model1a1 = trainer1a.fit(weightedData)
+ model1b = trainer1b.fit(weightedData)
+ }
+
+ assert(model1a0.weights !~= model1a1.weights absTol 1E-3)
+ assert(model1a0.intercept !~= model1a1.intercept absTol 1E-3)
+ assert(model1a0.weights ~== model1b.weights absTol 1E-3)
+ assert(model1a0.intercept ~== model1b.intercept absTol 1E-3)
+
+ val trainer2a = (new LinearRegression).setFitIntercept(true)
+ .setElasticNetParam(0.38).setRegParam(0.21).setStandardization(false).setSolver(solver)
+ val trainer2b = (new LinearRegression).setFitIntercept(true).setWeightCol("weight")
+ .setElasticNetParam(0.38).setRegParam(0.21).setStandardization(false).setSolver(solver)
+ val model2a0 = trainer2a.fit(data)
+ val model2a1 = trainer2a.fit(weightedData)
+ val model2b = trainer2b.fit(weightedData)
+ assert(model2a0.weights !~= model2a1.weights absTol 1E-3)
+ assert(model2a0.intercept !~= model2a1.intercept absTol 1E-3)
+ assert(model2a0.weights ~== model2b.weights absTol 1E-3)
+ assert(model2a0.intercept ~== model2b.intercept absTol 1E-3)
+
+ val trainer3a = (new LinearRegression).setFitIntercept(false)
+ .setElasticNetParam(0.38).setRegParam(0.21).setStandardization(true).setSolver(solver)
+ val trainer3b = (new LinearRegression).setFitIntercept(false).setWeightCol("weight")
+ .setElasticNetParam(0.38).setRegParam(0.21).setStandardization(true).setSolver(solver)
+ val model3a0 = trainer3a.fit(data)
+ val model3a1 = trainer3a.fit(weightedData)
+ val model3b = trainer3b.fit(weightedData)
+ assert(model3a0.weights !~= model3a1.weights absTol 1E-3)
+ assert(model3a0.weights ~== model3b.weights absTol 1E-3)
+
+ val trainer4a = (new LinearRegression).setFitIntercept(false)
+ .setElasticNetParam(0.38).setRegParam(0.21).setStandardization(false).setSolver(solver)
+ val trainer4b = (new LinearRegression).setFitIntercept(false).setWeightCol("weight")
+ .setElasticNetParam(0.38).setRegParam(0.21).setStandardization(false).setSolver(solver)
+ val model4a0 = trainer4a.fit(data)
+ val model4a1 = trainer4a.fit(weightedData)
+ val model4b = trainer4b.fit(weightedData)
+ assert(model4a0.weights !~= model4a1.weights absTol 1E-3)
+ assert(model4a0.weights ~== model4b.weights absTol 1E-3)
}
- val data2 = weightedSignedData ++ weightedNoiseData
-
- (sqlContext.createDataFrame(sc.parallelize(data1, 4)),
- sqlContext.createDataFrame(sc.parallelize(data2, 4)))
}
+ }
+
+ test("LBFGS and normal optimizer should not have diffs") {
--- End diff --
Why this test is added? I thought this one is already covered. What missing is that comparing weighted sample version of `normal` and `l-bfgs` solver. See the session of `test("linear regression with weighted samples"){`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-144388682
[Test build #43131 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43131/console) for PR 8884 at commit [`746cc5f`](https://github.com/apache/spark/commit/746cc5fdb16ff4ec265bd68efed9277c7b49d9c4).
* This patch **fails MiMa tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by Lewuathe <gi...@git.apache.org>.
Github user Lewuathe commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-146080738
@dbtsai Thank you so much for caching up. Currently this patch fixed the latest issues based on your pointing out. Test failures does not seem to be related to this patch. So could you check it?
And also I'm on abroad now to work other thing until 11th Oct, therefore it's difficult for me to update patch until 11th. If it is necessary to update further this before 11th, could you help me with it?
I'm very much so sorry for bothering you. But thanks again to reviewing this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147184510
[Test build #43542 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43542/consoleFull) for PR 8884 at commit [`7317aa2`](https://github.com/apache/spark/commit/7317aa255960a258e6b480188fe80a1e9454f461).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-143125346
[Test build #43007 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43007/console) for PR 8884 at commit [`78677dd`](https://github.com/apache/spark/commit/78677dda2e84a02eaa415347254ca446b5238a43).
* This patch **fails MiMa tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142578559
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-143185562
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43021/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-148691355
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43839/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142605070
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by Lewuathe <gi...@git.apache.org>.
Github user Lewuathe commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142769348
How can we try SparkR test at local? This seems also require update parameter of SparkR test code.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142790880
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42950/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41806842
--- Diff: R/pkg/R/mllib.R ---
@@ -45,12 +45,13 @@ setClass("PipelineModel", representation(model = "jobj"))
#' summary(model)
#'}
setMethod("glm", signature(formula = "formula", family = "ANY", data = "DataFrame"),
- function(formula, family = c("gaussian", "binomial"), data, lambda = 0, alpha = 0) {
- family <- match.arg(family)
- model <- callJStatic("org.apache.spark.ml.api.r.SparkRWrappers",
+ function(formula, family = c("gaussian", "binomial"), data, lambda = 0, alpha = 0,
+ solver = "auto") {
+ family <- match.arg(family)
+ model <- callJStatic("org.apache.spark.ml.api.r.SparkRWrappers",
"fitRModelFormula", deparse(formula), data@sdf, family, lambda,
- alpha)
- return(new("PipelineModel", model = model))
+ alpha, solver)
+ return(new("PipelineModel", model = model))
--- End diff --
ditto.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by Lewuathe <gi...@git.apache.org>.
Github user Lewuathe commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40743997
--- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -169,6 +181,33 @@ class LinearRegression(override val uid: String)
val yMean = ySummarizer.mean(0)
val yStd = math.sqrt(ySummarizer.variance(0))
--- End diff --
Currently `numFeatures` is obtained by summarization. In order to decide which algorithm is used for "auto" solver, it is necessary to get `numFeatures`. Is is ok to obtain `numFeatures` from first instance on dataset?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41809032
--- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -130,9 +131,54 @@ class LinearRegression(override val uid: String)
def setWeightCol(value: String): this.type = set(weightCol, value)
setDefault(weightCol -> "")
+ /**
+ * Set the solver algorithm used for optimization.
+ * In case of linear regression, this can be "l-bfgs", "normal" and "auto".
+ * The default value is "auto" which means that the solver algorithm is
+ * selected automatically.
+ * @group setParam
+ */
+ def setSolver(value: String): this.type = set(solver, value)
+ setDefault(solver -> "auto")
+
override protected def train(dataset: DataFrame): LinearRegressionModel = {
+ // Extract the number of features before deciding optimization solver.
+ val numFeatures = dataset.select(col($(featuresCol))).limit(1).map {
+ case Row(features: Vector) =>
+ features.size
+ }.toArray()(0)
// Extract columns from data. If dataset is persisted, do not persist instances.
val w = if ($(weightCol).isEmpty) lit(1.0) else col($(weightCol))
+
+ if ($(solver) == "normal" || ($(solver) == "auto"
+ && $(elasticNetParam) == 0.0 && numFeatures <= 4096)) {
+ require($(elasticNetParam) == 0.0, "Only L2 regularization can be used when normal " +
+ "solver is selected.'")
+ // In case of feature size is small, WeightedLeastSquares can train more efficiently
+ // because it requires one pass through to the data. (SPARK-10668)
--- End diff --
For low dimensional data, WeightedLeastSquares is more efficiently since the training algorithm only requires one pass through the data.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-144388724
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43131/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-142605072
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42907/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-143132517
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40518673
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -302,29 +314,31 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
as.numeric.data.V2. 3.435466
as.numeric.data.V3. 5.910406
*/
- val interceptR2 = 5.791109
- val weightsR2 = Vectors.dense(3.435466, 5.910406)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ val interceptR2 = 5.791109
+ val weightsR2 = Vectors.dense(3.435466, 5.910406)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ })
}
test("linear regression without intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
+ Seq("auto", "l-bfgs", "normal").foreach(solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
--- End diff --
ditto
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40518573
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -166,22 +171,25 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
as.numeric.data3.V2. 4.70011
as.numeric.data3.V3. 7.19943
*/
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ })
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
+ Seq("auto").foreach(solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
--- End diff --
ditto.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147726706
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43643/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147546487
@Lewuathe LGTM except those minor issues.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by yanboliang <gi...@git.apache.org>.
Github user yanboliang commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40529712
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -150,14 +155,14 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
as.numeric.data.V2. 6.995908
as.numeric.data.V3. 5.275131
*/
- val weightsR = Vectors.dense(6.995908, 5.275131)
+ val weightsR = Vectors.dense(6.995908, 5.275131)
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
+ assert(model1.intercept ~== 0 absTol 1E-3)
--- End diff --
Lots of indent issues you should fix.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by Lewuathe <gi...@git.apache.org>.
Github user Lewuathe commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40528709
--- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -199,8 +238,8 @@ class LinearRegression(override val uid: String)
val costFun = new LeastSquaresCostFun(instances, yStd, yMean, $(fitIntercept),
$(standardization), featuresStd, featuresMean, effectiveL2RegParam)
-
- val optimizer = if ($(elasticNetParam) == 0.0 || effectiveRegParam == 0.0) {
--- End diff --
Otherwise, there may be the case when `BreezeOWLQN` is used even if solver is set "l-bfgs".
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by Lewuathe <gi...@git.apache.org>.
Github user Lewuathe commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41856388
--- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala ---
@@ -130,9 +131,54 @@ class LinearRegression(override val uid: String)
def setWeightCol(value: String): this.type = set(weightCol, value)
setDefault(weightCol -> "")
+ /**
+ * Set the solver algorithm used for optimization.
+ * In case of linear regression, this can be "l-bfgs", "normal" and "auto".
+ * The default value is "auto" which means that the solver algorithm is
+ * selected automatically.
+ * @group setParam
+ */
+ def setSolver(value: String): this.type = set(solver, value)
+ setDefault(solver -> "auto")
+
override protected def train(dataset: DataFrame): LinearRegressionModel = {
+ // Extract the number of features before deciding optimization solver.
+ val numFeatures = dataset.select(col($(featuresCol))).limit(1).map {
+ case Row(features: Vector) =>
+ features.size
+ }.toArray()(0)
// Extract columns from data. If dataset is persisted, do not persist instances.
val w = if ($(weightCol).isEmpty) lit(1.0) else col($(weightCol))
+
+ if ($(solver) == "normal" || ($(solver) == "auto"
+ && $(elasticNetParam) == 0.0 && numFeatures <= 4096)) {
+ require($(elasticNetParam) == 0.0, "Only L2 regularization can be used when normal " +
+ "solver is selected.'")
+ // In case of feature size is small, WeightedLeastSquares can train more efficiently
+ // because it requires one pass through to the data. (SPARK-10668)
+ val instances: RDD[WeightedLeastSquares.Instance] = dataset.select(
+ col($(labelCol)), w, col($(featuresCol))).map {
+ case Row(label: Double, weight: Double, features: Vector) =>
+ WeightedLeastSquares.Instance(weight, features, label)
+ }
+
+ val optimizer = new WeightedLeastSquares($(fitIntercept), $(regParam),
+ $(standardization), true)
+ val model = optimizer.fit(instances)
+ // When it is trained by WeightedLeastSquares, training summary does not
--- End diff --
Yes, it is still true. Because `WeightedLeastSquares` does not generate a objective history, an only one element array is added to summary.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147182508
[Test build #43541 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43541/console) for PR 8884 at commit [`99916cc`](https://github.com/apache/spark/commit/99916cc1ade4202ba32862aa11b6c2f42f7daa2b).
* This patch **fails Scala style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-143132509
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by Lewuathe <gi...@git.apache.org>.
Github user Lewuathe commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r42338873
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +96,641 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with only L1 regularization case.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ break
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+ }
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.24300
+ as.numeric.data.V2. 4.024821
+ as.numeric.data.V3. 6.679841
+ */
+ val interceptR1 = 6.24300
+ val weightsR1 = Vectors.dense(4.024821, 6.679841)
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.416948
+ as.numeric.data.V2. 3.893869
+ as.numeric.data.V3. 6.724286
+ */
+ val interceptR2 = 6.416948
+ val weightsR2 = Vectors.dense(3.893869, 6.724286)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression without intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.299752
- as.numeric.data.V3. 4.772913
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(6.299752, 4.772913)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.232193
- as.numeric.data.V3. 4.764229
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(6.232193, 4.764229)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with only L1 regularization case.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ break
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+ }
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.299752
+ as.numeric.data.V3. 4.772913
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(6.299752, 4.772913)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.232193
+ as.numeric.data.V3. 4.764229
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(6.232193, 4.764229)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression with intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.269376
- as.numeric.data.V2. 3.736216
- as.numeric.data.V3. 5.712356)
- */
- val interceptR1 = 5.269376
- val weightsR1 = Vectors.dense(3.736216, 5.712356)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.791109
- as.numeric.data.V2. 3.435466
- as.numeric.data.V3. 5.910406
- */
- val interceptR2 = 5.791109
- val weightsR2 = Vectors.dense(3.435466, 5.910406)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.269376
+ as.numeric.data.V2. 3.736216
+ as.numeric.data.V3. 5.712356)
+ */
+ val interceptR1 = 5.269376
+ val weightsR1 = Vectors.dense(3.736216, 5.712356)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.791109
+ as.numeric.data.V2. 3.435466
+ as.numeric.data.V3. 5.910406
+ */
+ val interceptR2 = 5.791109
+ val weightsR2 = Vectors.dense(3.435466, 5.910406)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression without intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.522875
- as.numeric.data.V3. 4.214502
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.522875, 4.214502)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.263704
- as.numeric.data.V3. 4.187419
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.263704, 4.187419)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.522875
+ as.numeric.data.V3. 4.214502
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.522875, 4.214502)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.263704
+ as.numeric.data.V3. 4.187419
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.263704, 4.187419)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
}
}
test("linear regression with intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.324108
- as.numeric.data.V2. 3.168435
- as.numeric.data.V3. 5.200403
- */
- val interceptR1 = 5.696056
- val weightsR1 = Vectors.dense(3.670489, 6.001122)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.114723
- as.numeric.data.V2. 3.409937
- as.numeric.data.V3. 6.146531
- */
- val interceptR2 = 6.114723
- val weightsR2 = Vectors.dense(3.409937, 6.146531)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setStandardization(false).setSolver(solver)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with non-zero elasticnet parameter.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ break
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+ }
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.324108
+ as.numeric.data.V2. 3.168435
+ as.numeric.data.V3. 5.200403
+ */
+ val interceptR1 = 5.696056
+ val weightsR1 = Vectors.dense(3.670489, 6.001122)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.114723
+ as.numeric.data.V2. 3.409937
+ as.numeric.data.V3. 6.146531
+ */
+ val interceptR2 = 6.114723
+ val weightsR2 = Vectors.dense(3.409937, 6.146531)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression without intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.dataM.V2. 5.673348
- as.numeric.dataM.V3. 4.322251
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.673348, 4.322251)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.477988
- as.numeric.data.V3. 4.297622
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.477988, 4.297622)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ var model1: LinearRegressionModel = null
+ var model2: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with non-zero elasticnet parameter.
+ if (solver == "normal") {
+ intercept[IllegalArgumentException] {
+ trainer1.fit(dataset)
+ trainer2.fit(dataset)
+ }
+ break
+ } else {
+ model1 = trainer1.fit(dataset)
+ model2 = trainer2.fit(dataset)
+ }
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.dataM.V2. 5.673348
+ as.numeric.dataM.V3. 4.322251
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.673348, 4.322251)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.477988
+ as.numeric.data.V3. 4.297622
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.477988, 4.297622)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }
}
}
test("linear regression model training summary") {
- val trainer = new LinearRegression
- val model = trainer.fit(dataset)
- val trainerNoPredictionCol = trainer.setPredictionCol("")
- val modelNoPredictionCol = trainerNoPredictionCol.fit(dataset)
-
-
- // Training results for the model should be available
- assert(model.hasSummary)
- assert(modelNoPredictionCol.hasSummary)
-
- // Schema should be a superset of the input dataset
- assert((dataset.schema.fieldNames.toSet + "prediction").subsetOf(
- model.summary.predictions.schema.fieldNames.toSet))
- // Validate that we re-insert a prediction column for evaluation
- val modelNoPredictionColFieldNames = modelNoPredictionCol.summary.predictions.schema.fieldNames
- assert((dataset.schema.fieldNames.toSet).subsetOf(
- modelNoPredictionColFieldNames.toSet))
- assert(modelNoPredictionColFieldNames.exists(s => s.startsWith("prediction_")))
-
- // Residuals in [[LinearRegressionResults]] should equal those manually computed
- val expectedResiduals = dataset.select("features", "label")
- .map { case Row(features: DenseVector, label: Double) =>
- val prediction =
- features(0) * model.weights(0) + features(1) * model.weights(1) + model.intercept
- label - prediction
- }
- .zip(model.summary.residuals.map(_.getDouble(0)))
- .collect()
- .foreach { case (manualResidual: Double, resultResidual: Double) =>
- assert(manualResidual ~== resultResidual relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer = new LinearRegression().setSolver(solver)
+ val model = trainer.fit(dataset)
+ val trainerNoPredictionCol = trainer.setPredictionCol("")
+ val modelNoPredictionCol = trainerNoPredictionCol.fit(dataset)
+
+
+ // Training results for the model should be available
+ assert(model.hasSummary)
+ assert(modelNoPredictionCol.hasSummary)
+
+ // Schema should be a superset of the input dataset
+ assert((dataset.schema.fieldNames.toSet + "prediction").subsetOf(
+ model.summary.predictions.schema.fieldNames.toSet))
+ // Validate that we re-insert a prediction column for evaluation
+ val modelNoPredictionColFieldNames
+ = modelNoPredictionCol.summary.predictions.schema.fieldNames
+ assert((dataset.schema.fieldNames.toSet).subsetOf(
+ modelNoPredictionColFieldNames.toSet))
+ assert(modelNoPredictionColFieldNames.exists(s => s.startsWith("prediction_")))
+
+ // Residuals in [[LinearRegressionResults]] should equal those manually computed
+ val expectedResiduals = dataset.select("features", "label")
+ .map { case Row(features: DenseVector, label: Double) =>
+ val prediction =
+ features(0) * model.weights(0) + features(1) * model.weights(1) + model.intercept
+ label - prediction
+ }
+ .zip(model.summary.residuals.map(_.getDouble(0)))
+ .collect()
+ .foreach { case (manualResidual: Double, resultResidual: Double) =>
+ assert(manualResidual ~== resultResidual relTol 1E-5)
+ }
- /*
- Use the following R code to generate model training results.
-
- predictions <- predict(fit, newx=features)
- residuals <- label - predictions
- > mean(residuals^2) # MSE
- [1] 0.009720325
- > mean(abs(residuals)) # MAD
- [1] 0.07863206
- > cor(predictions, label)^2# r^2
- [,1]
- s0 0.9998749
- */
- assert(model.summary.meanSquaredError ~== 0.00972035 relTol 1E-5)
- assert(model.summary.meanAbsoluteError ~== 0.07863206 relTol 1E-5)
- assert(model.summary.r2 ~== 0.9998749 relTol 1E-5)
-
- // Objective function should be monotonically decreasing for linear regression
- assert(
- model.summary
- .objectiveHistory
- .sliding(2)
- .forall(x => x(0) >= x(1)))
+ /*
+ Use the following R code to generate model training results.
+
+ predictions <- predict(fit, newx=features)
+ residuals <- label - predictions
+ > mean(residuals^2) # MSE
+ [1] 0.009720325
+ > mean(abs(residuals)) # MAD
+ [1] 0.07863206
+ > cor(predictions, label)^2# r^2
+ [,1]
+ s0 0.9998749
+ */
+ assert(model.summary.meanSquaredError ~== 0.00972035 relTol 1E-5)
+ assert(model.summary.meanAbsoluteError ~== 0.07863206 relTol 1E-5)
+ assert(model.summary.r2 ~== 0.9998749 relTol 1E-5)
+
+ // Normal solver uses "WeightedLeastSquares". This algorithm does not generate
+ // objective history because it does not run through iterations.
+ if (solver == "l-bfgs") {
+ // Objective function should be monotonically decreasing for linear regression
+ assert(
+ model.summary
+ .objectiveHistory
+ .sliding(2)
+ .forall(x => x(0) >= x(1)))
+ }
+ }
}
test("linear regression model testset evaluation summary") {
- val trainer = new LinearRegression
- val model = trainer.fit(dataset)
-
- // Evaluating on training dataset should yield results summary equal to training summary
- val testSummary = model.evaluate(dataset)
- assert(model.summary.meanSquaredError ~== testSummary.meanSquaredError relTol 1E-5)
- assert(model.summary.r2 ~== testSummary.r2 relTol 1E-5)
- model.summary.residuals.select("residuals").collect()
- .zip(testSummary.residuals.select("residuals").collect())
- .forall { case (Row(r1: Double), Row(r2: Double)) => r1 ~== r2 relTol 1E-5 }
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val trainer = new LinearRegression().setSolver(solver)
+ val model = trainer.fit(dataset)
+
+ // Evaluating on training dataset should yield results summary equal to training summary
+ val testSummary = model.evaluate(dataset)
+ assert(model.summary.meanSquaredError ~== testSummary.meanSquaredError relTol 1E-5)
+ assert(model.summary.r2 ~== testSummary.r2 relTol 1E-5)
+ model.summary.residuals.select("residuals").collect()
+ .zip(testSummary.residuals.select("residuals").collect())
+ .forall { case (Row(r1: Double), Row(r2: Double)) => r1 ~== r2 relTol 1E-5 }
+ }
}
test("linear regression with weighted samples"){
- val (data, weightedData) = {
- val activeData = LinearDataGenerator.generateLinearInput(
- 6.3, Array(4.7, 7.2), Array(0.9, -1.3), Array(0.7, 1.2), 500, 1, 0.1)
-
- val rnd = new Random(8392)
- val signedData = activeData.map { case p: LabeledPoint =>
- (rnd.nextGaussian() > 0.0, p)
- }
-
- val data1 = signedData.flatMap {
- case (true, p) => Iterator(p, p)
- case (false, p) => Iterator(p)
- }
-
- val weightedSignedData = signedData.flatMap {
- case (true, LabeledPoint(label, features)) =>
- Iterator(
- Instance(label, weight = 1.2, features),
- Instance(label, weight = 0.8, features)
- )
- case (false, LabeledPoint(label, features)) =>
- Iterator(
- Instance(label, weight = 0.3, features),
- Instance(label, weight = 0.1, features),
- Instance(label, weight = 0.6, features)
- )
- }
-
- val noiseData = LinearDataGenerator.generateLinearInput(
- 2, Array(1, 3), Array(0.9, -1.3), Array(0.7, 1.2), 500, 1, 0.1)
- val weightedNoiseData = noiseData.map {
- case LabeledPoint(label, features) => Instance(label, weight = 0, features)
+ breakable {
+ Seq("auto", "l-bfgs", "normal").foreach { solver =>
+ val (data, weightedData) = {
+ val activeData = LinearDataGenerator.generateLinearInput(
+ 6.3, Array(4.7, 7.2), Array(0.9, -1.3), Array(0.7, 1.2), 500, 1, 0.1)
+
+ val rnd = new Random(8392)
+ val signedData = activeData.map { case p: LabeledPoint =>
+ (rnd.nextGaussian() > 0.0, p)
+ }
+
+ val data1 = signedData.flatMap {
+ case (true, p) => Iterator(p, p)
+ case (false, p) => Iterator(p)
+ }
+
+ val weightedSignedData = signedData.flatMap {
+ case (true, LabeledPoint(label, features)) =>
+ Iterator(
+ Instance(label, weight = 1.2, features),
+ Instance(label, weight = 0.8, features)
+ )
+ case (false, LabeledPoint(label, features)) =>
+ Iterator(
+ Instance(label, weight = 0.3, features),
+ Instance(label, weight = 0.1, features),
+ Instance(label, weight = 0.6, features)
+ )
+ }
+
+ val noiseData = LinearDataGenerator.generateLinearInput(
+ 2, Array(1, 3), Array(0.9, -1.3), Array(0.7, 1.2), 500, 1, 0.1)
+ val weightedNoiseData = noiseData.map {
+ case LabeledPoint(label, features) => Instance(label, weight = 0, features)
+ }
+ val data2 = weightedSignedData ++ weightedNoiseData
+
+ (sqlContext.createDataFrame(sc.parallelize(data1, 4)),
+ sqlContext.createDataFrame(sc.parallelize(data2, 4)))
+ }
+
+ val trainer1a = (new LinearRegression).setFitIntercept(true)
+ .setElasticNetParam(0.38).setRegParam(0.21).setStandardization(true).setSolver(solver)
+ val trainer1b = (new LinearRegression).setFitIntercept(true).setWeightCol("weight")
+ .setElasticNetParam(0.38).setRegParam(0.21).setStandardization(true).setSolver(solver)
+
+ var model1a0: LinearRegressionModel = null
+ var model1a1: LinearRegressionModel = null
+ var model1b: LinearRegressionModel = null
+
+ // Normal optimizer is not supported with non-zero elasticnet parameter.
+ if (solver == "normal") {
--- End diff --
But in this test case, `elasticNetParam` does not 0.0 actually 0.38. So do you mean we should change this parameter to 0.0 inside the "linear regression with weighted samples" test case?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41811080
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,559 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }}
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.24300
+ as.numeric.data.V2. 4.024821
+ as.numeric.data.V3. 6.679841
+ */
+ val interceptR1 = 6.24300
+ val weightsR1 = Vectors.dense(4.024821, 6.679841)
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.416948
+ as.numeric.data.V2. 3.893869
+ as.numeric.data.V3. 6.724286
+ */
+ val interceptR2 = 6.416948
+ val weightsR2 = Vectors.dense(3.893869, 6.724286)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.299752
- as.numeric.data.V3. 4.772913
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(6.299752, 4.772913)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.232193
- as.numeric.data.V3. 4.764229
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(6.232193, 4.764229)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.299752
+ as.numeric.data.V3. 4.772913
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(6.299752, 4.772913)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.232193
+ as.numeric.data.V3. 4.764229
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(6.232193, 4.764229)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
--- End diff --
ditto
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r40518641
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -206,30 +213,32 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
as.numeric.data.V2. 3.893869
as.numeric.data.V3. 6.724286
*/
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
+ val interceptR2 = 6.416948
+ val weightsR2 = Vectors.dense(3.893869, 6.724286)
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ })
}
test("linear regression without intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
--- End diff --
Maybe you can still do `Seq("auto", "l-bfgs", "normal").foreach`, but when solver is `normal`, you do `intercept` for `exception`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by Lewuathe <gi...@git.apache.org>.
Github user Lewuathe commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147712068
@dbtsai Thank you so much for reviewing many times. Could check it again, please?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-147919639
[Test build #43699 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43699/console) for PR 8884 at commit [`ea5f195`](https://github.com/apache/spark/commit/ea5f195f8b3fce7dac5b57750ef44bb59c9a0eb3).
* This patch **fails MiMa tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on the pull request:
https://github.com/apache/spark/pull/8884#issuecomment-149671660
@Lewuathe Can you create the JIRA with detail? Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by Lewuathe <gi...@git.apache.org>.
Github user Lewuathe commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41863564
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,559 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }}
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setSolver(solver).setStandardization(false)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.24300
+ as.numeric.data.V2. 4.024821
+ as.numeric.data.V3. 6.679841
+ */
+ val interceptR1 = 6.24300
+ val weightsR1 = Vectors.dense(4.024821, 6.679841)
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.416948
+ as.numeric.data.V2. 3.893869
+ as.numeric.data.V3. 6.724286
+ */
+ val interceptR2 = 6.416948
+ val weightsR2 = Vectors.dense(3.893869, 6.724286)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.299752
- as.numeric.data.V3. 4.772913
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(6.299752, 4.772913)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.232193
- as.numeric.data.V3. 4.764229
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(6.232193, 4.764229)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.299752
+ as.numeric.data.V3. 4.772913
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(6.299752, 4.772913)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.232193
+ as.numeric.data.V3. 4.764229
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(6.232193, 4.764229)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression with intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.269376
- as.numeric.data.V2. 3.736216
- as.numeric.data.V3. 5.712356)
- */
- val interceptR1 = 5.269376
- val weightsR1 = Vectors.dense(3.736216, 5.712356)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 5.791109
- as.numeric.data.V2. 3.435466
- as.numeric.data.V3. 5.910406
- */
- val interceptR2 = 5.791109
- val weightsR2 = Vectors.dense(3.435466, 5.910406)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.269376
+ as.numeric.data.V2. 3.736216
+ as.numeric.data.V3. 5.712356)
+ */
+ val interceptR1 = 5.269376
+ val weightsR1 = Vectors.dense(3.736216, 5.712356)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 5.791109
+ as.numeric.data.V2. 3.435466
+ as.numeric.data.V3. 5.910406
+ */
+ val interceptR2 = 5.791109
+ val weightsR2 = Vectors.dense(3.435466, 5.910406)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept with L2 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.522875
- as.numeric.data.V3. 4.214502
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.522875, 4.214502)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
- intercept = FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.263704
- as.numeric.data.V3. 4.187419
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.263704, 4.187419)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.0).setRegParam(2.3)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.522875
+ as.numeric.data.V3. 4.214502
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.522875, 4.214502)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.0, lambda = 2.3,
+ intercept = FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.263704
+ as.numeric.data.V3. 4.187419
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.263704, 4.187419)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression with intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.324108
- as.numeric.data.V2. 3.168435
- as.numeric.data.V3. 5.200403
- */
- val interceptR1 = 5.696056
- val weightsR1 = Vectors.dense(3.670489, 6.001122)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.114723
- as.numeric.data.V2. 3.409937
- as.numeric.data.V3. 6.146531
- */
- val interceptR2 = 6.114723
- val weightsR2 = Vectors.dense(3.409937, 6.146531)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.324108
+ as.numeric.data.V2. 3.168435
+ as.numeric.data.V3. 5.200403
+ */
+ val interceptR1 = 5.696056
+ val weightsR1 = Vectors.dense(3.670489, 6.001122)
+
+ assert(model1.intercept ~== interceptR1 relTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6
+ standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.114723
+ as.numeric.data.V2. 3.409937
+ as.numeric.data.V3. 6.146531
+ */
+ val interceptR2 = 6.114723
+ val weightsR2 = Vectors.dense(3.409937, 6.146531)
+
+ assert(model2.intercept ~== interceptR2 relTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept with ElasticNet regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false)
- val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
- .setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.dataM.V2. 5.673348
- as.numeric.dataM.V3. 4.322251
- */
- val interceptR1 = 0.0
- val weightsR1 = Vectors.dense(5.673348, 4.322251)
-
- assert(model1.intercept ~== interceptR1 absTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
- intercept=FALSE, standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 5.477988
- as.numeric.data.V3. 4.297622
- */
- val interceptR2 = 0.0
- val weightsR2 = Vectors.dense(5.477988, 4.297622)
-
- assert(model2.intercept ~== interceptR2 absTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
+ val trainer1 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setFitIntercept(false).setSolver(solver)
+ val trainer2 = (new LinearRegression).setElasticNetParam(0.3).setRegParam(1.6)
+ .setFitIntercept(false).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
+ intercept=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.dataM.V2. 5.673348
+ as.numeric.dataM.V3. 4.322251
+ */
+ val interceptR1 = 0.0
+ val weightsR1 = Vectors.dense(5.673348, 4.322251)
+
+ assert(model1.intercept ~== interceptR1 absTol 1E-3)
+ assert(model1.weights ~= weightsR1 relTol 1E-3)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0.3, lambda = 1.6,
+ intercept=FALSE, standardize=FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 5.477988
+ as.numeric.data.V3. 4.297622
+ */
+ val interceptR2 = 0.0
+ val weightsR2 = Vectors.dense(5.477988, 4.297622)
+
+ assert(model2.intercept ~== interceptR2 absTol 1E-3)
+ assert(model2.weights ~= weightsR2 relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression model training summary") {
- val trainer = new LinearRegression
- val model = trainer.fit(dataset)
- val trainerNoPredictionCol = trainer.setPredictionCol("")
- val modelNoPredictionCol = trainerNoPredictionCol.fit(dataset)
-
-
- // Training results for the model should be available
- assert(model.hasSummary)
- assert(modelNoPredictionCol.hasSummary)
-
- // Schema should be a superset of the input dataset
- assert((dataset.schema.fieldNames.toSet + "prediction").subsetOf(
- model.summary.predictions.schema.fieldNames.toSet))
- // Validate that we re-insert a prediction column for evaluation
- val modelNoPredictionColFieldNames = modelNoPredictionCol.summary.predictions.schema.fieldNames
- assert((dataset.schema.fieldNames.toSet).subsetOf(
- modelNoPredictionColFieldNames.toSet))
- assert(modelNoPredictionColFieldNames.exists(s => s.startsWith("prediction_")))
-
- // Residuals in [[LinearRegressionResults]] should equal those manually computed
- val expectedResiduals = dataset.select("features", "label")
- .map { case Row(features: DenseVector, label: Double) =>
- val prediction =
- features(0) * model.weights(0) + features(1) * model.weights(1) + model.intercept
- label - prediction
- }
- .zip(model.summary.residuals.map(_.getDouble(0)))
- .collect()
- .foreach { case (manualResidual: Double, resultResidual: Double) =>
- assert(manualResidual ~== resultResidual relTol 1E-5)
- }
+ Seq("l-bfgs").foreach { solver => {
+ val trainer = new LinearRegression().setSolver(solver)
+ val model = trainer.fit(dataset)
+ val trainerNoPredictionCol = trainer.setPredictionCol("")
+ val modelNoPredictionCol = trainerNoPredictionCol.fit(dataset)
+
+
+ // Training results for the model should be available
+ assert(model.hasSummary)
+ assert(modelNoPredictionCol.hasSummary)
+
+ // Schema should be a superset of the input dataset
+ assert((dataset.schema.fieldNames.toSet + "prediction").subsetOf(
+ model.summary.predictions.schema.fieldNames.toSet))
+ // Validate that we re-insert a prediction column for evaluation
+ val modelNoPredictionColFieldNames
+ = modelNoPredictionCol.summary.predictions.schema.fieldNames
+ assert((dataset.schema.fieldNames.toSet).subsetOf(
+ modelNoPredictionColFieldNames.toSet))
+ assert(modelNoPredictionColFieldNames.exists(s => s.startsWith("prediction_")))
+
+ // Residuals in [[LinearRegressionResults]] should equal those manually computed
+ val expectedResiduals = dataset.select("features", "label")
+ .map { case Row(features: DenseVector, label: Double) =>
+ val prediction =
+ features(0) * model.weights(0) + features(1) * model.weights(1) + model.intercept
+ label - prediction
+ }
+ .zip(model.summary.residuals.map(_.getDouble(0)))
+ .collect()
+ .foreach { case (manualResidual: Double, resultResidual: Double) =>
+ assert(manualResidual ~== resultResidual relTol 1E-5)
+ }
- /*
- Use the following R code to generate model training results.
-
- predictions <- predict(fit, newx=features)
- residuals <- label - predictions
- > mean(residuals^2) # MSE
- [1] 0.009720325
- > mean(abs(residuals)) # MAD
- [1] 0.07863206
- > cor(predictions, label)^2# r^2
- [,1]
- s0 0.9998749
- */
- assert(model.summary.meanSquaredError ~== 0.00972035 relTol 1E-5)
- assert(model.summary.meanAbsoluteError ~== 0.07863206 relTol 1E-5)
- assert(model.summary.r2 ~== 0.9998749 relTol 1E-5)
-
- // Objective function should be monotonically decreasing for linear regression
- assert(
- model.summary
- .objectiveHistory
- .sliding(2)
- .forall(x => x(0) >= x(1)))
+ /*
+ Use the following R code to generate model training results.
+
+ predictions <- predict(fit, newx=features)
+ residuals <- label - predictions
+ > mean(residuals^2) # MSE
+ [1] 0.009720325
+ > mean(abs(residuals)) # MAD
+ [1] 0.07863206
+ > cor(predictions, label)^2# r^2
+ [,1]
+ s0 0.9998749
+ */
+ assert(model.summary.meanSquaredError ~== 0.00972035 relTol 1E-5)
+ assert(model.summary.meanAbsoluteError ~== 0.07863206 relTol 1E-5)
+ assert(model.summary.r2 ~== 0.9998749 relTol 1E-5)
+
+ // Normal solver uses "WeightedLeastSquares". This algorithm does not generate
+ // objective history because it does not run through iterations.
+ if (solver == "l-bfgs") {
+ // Objective function should be monotonically decreasing for linear regression
+ assert(
+ model.summary
+ .objectiveHistory
+ .sliding(2)
+ .forall(x => x(0) >= x(1)))
+ }
+ }}
}
test("linear regression model testset evaluation summary") {
- val trainer = new LinearRegression
- val model = trainer.fit(dataset)
-
- // Evaluating on training dataset should yield results summary equal to training summary
- val testSummary = model.evaluate(dataset)
- assert(model.summary.meanSquaredError ~== testSummary.meanSquaredError relTol 1E-5)
- assert(model.summary.r2 ~== testSummary.r2 relTol 1E-5)
- model.summary.residuals.select("residuals").collect()
- .zip(testSummary.residuals.select("residuals").collect())
- .forall { case (Row(r1: Double), Row(r2: Double)) => r1 ~== r2 relTol 1E-5 }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer = new LinearRegression().setSolver(solver)
+ val model = trainer.fit(dataset)
+
+ // Evaluating on training dataset should yield results summary equal to training summary
+ val testSummary = model.evaluate(dataset)
+ assert(model.summary.meanSquaredError ~== testSummary.meanSquaredError relTol 1E-5)
+ assert(model.summary.r2 ~== testSummary.r2 relTol 1E-5)
+ model.summary.residuals.select("residuals").collect()
+ .zip(testSummary.residuals.select("residuals").collect())
+ .forall { case (Row(r1: Double), Row(r2: Double)) => r1 ~== r2 relTol 1E-5 }
+ }}
}
test("linear regression with weighted samples"){
- val (data, weightedData) = {
- val activeData = LinearDataGenerator.generateLinearInput(
- 6.3, Array(4.7, 7.2), Array(0.9, -1.3), Array(0.7, 1.2), 500, 1, 0.1)
-
- val rnd = new Random(8392)
- val signedData = activeData.map { case p: LabeledPoint =>
- (rnd.nextGaussian() > 0.0, p)
- }
-
- val data1 = signedData.flatMap {
- case (true, p) => Iterator(p, p)
- case (false, p) => Iterator(p)
- }
-
- val weightedSignedData = signedData.flatMap {
- case (true, LabeledPoint(label, features)) =>
- Iterator(
- Instance(label, weight = 1.2, features),
- Instance(label, weight = 0.8, features)
- )
- case (false, LabeledPoint(label, features)) =>
- Iterator(
- Instance(label, weight = 0.3, features),
- Instance(label, weight = 0.1, features),
- Instance(label, weight = 0.6, features)
- )
+ Seq("auto", "l-bfgs").foreach { solver => {
--- End diff --
This test case includes elastic parameter. Then it occurres `IllegalArgumentException`.
And then this should be handled with `intercept`. But it becomes a little hacky to write this type test code into a single test as previous mentioned.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10668][ML] Use WeightedLeastSquares in ...
Posted by dbtsai <gi...@git.apache.org>.
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/8884#discussion_r41810586
--- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala ---
@@ -93,525 +94,559 @@ class LinearRegressionSuite extends SparkFunSuite with MLlibTestSparkContext {
}
test("linear regression with intercept without regularization") {
- val trainer1 = new LinearRegression
- // The result should be the same regardless of standardization without regularization
- val trainer2 = (new LinearRegression).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- Using the following R code to load the data and train the model using glmnet package.
-
- library("glmnet")
- data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
- features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
- label <- as.numeric(data$V1)
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.298698
- as.numeric.data.V2. 4.700706
- as.numeric.data.V3. 7.199082
- */
- val interceptR = 6.298698
- val weightsR = Vectors.dense(4.700706, 7.199082)
-
- assert(model1.intercept ~== interceptR relTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== interceptR relTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = new LinearRegression().setSolver(solver)
+ // The result should be the same regardless of standardization without regularization
+ val trainer2 = (new LinearRegression).setStandardization(false).setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val model2 = trainer2.fit(dataset)
+
+ /*
+ Using the following R code to load the data and train the model using glmnet package.
+
+ library("glmnet")
+ data <- read.csv("path", header=FALSE, stringsAsFactors=FALSE)
+ features <- as.matrix(data.frame(as.numeric(data$V2), as.numeric(data$V3)))
+ label <- as.numeric(data$V1)
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) 6.298698
+ as.numeric.data.V2. 4.700706
+ as.numeric.data.V3. 7.199082
+ */
+ val interceptR = 6.298698
+ val weightsR = Vectors.dense(4.700706, 7.199082)
+
+ assert(model1.intercept ~== interceptR relTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== interceptR relTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ model1.transform(dataset).select("features", "prediction").collect().foreach {
+ case Row(features: DenseVector, prediction1: Double) =>
+ val prediction2 =
+ features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
+ assert(prediction1 ~== prediction2 relTol 1E-5)
+ }
+ }}
}
test("linear regression without intercept without regularization") {
- val trainer1 = (new LinearRegression).setFitIntercept(false)
- // Without regularization the results should be the same
- val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
- val model2 = trainer2.fit(dataset)
- val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
-
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
- intercept = FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data.V2. 6.995908
- as.numeric.data.V3. 5.275131
- */
- val weightsR = Vectors.dense(6.995908, 5.275131)
-
- assert(model1.intercept ~== 0 absTol 1E-3)
- assert(model1.weights ~= weightsR relTol 1E-3)
- assert(model2.intercept ~== 0 absTol 1E-3)
- assert(model2.weights ~= weightsR relTol 1E-3)
-
- /*
- Then again with the data with no intercept:
- > weightsWithoutIntercept
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) .
- as.numeric.data3.V2. 4.70011
- as.numeric.data3.V3. 7.19943
- */
- val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
-
- assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
- assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
- assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ Seq("auto", "l-bfgs", "normal").foreach { solver => {
+ val trainer1 = (new LinearRegression).setFitIntercept(false).setSolver(solver)
+ // Without regularization the results should be the same
+ val trainer2 = (new LinearRegression).setFitIntercept(false).setStandardization(false)
+ .setSolver(solver)
+ val model1 = trainer1.fit(dataset)
+ val modelWithoutIntercept1 = trainer1.fit(datasetWithoutIntercept)
+ val model2 = trainer2.fit(dataset)
+ val modelWithoutIntercept2 = trainer2.fit(datasetWithoutIntercept)
+
+ /*
+ weights <- coef(glmnet(features, label, family="gaussian", alpha = 0, lambda = 0,
+ intercept = FALSE))
+ > weights
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data.V2. 6.995908
+ as.numeric.data.V3. 5.275131
+ */
+ val weightsR = Vectors.dense(6.995908, 5.275131)
+
+ assert(model1.intercept ~== 0 absTol 1E-3)
+ assert(model1.weights ~= weightsR relTol 1E-3)
+ assert(model2.intercept ~== 0 absTol 1E-3)
+ assert(model2.weights ~= weightsR relTol 1E-3)
+
+ /*
+ Then again with the data with no intercept:
+ > weightsWithoutIntercept
+ 3 x 1 sparse Matrix of class "dgCMatrix"
+ s0
+ (Intercept) .
+ as.numeric.data3.V2. 4.70011
+ as.numeric.data3.V3. 7.19943
+ */
+ val weightsWithoutInterceptR = Vectors.dense(4.70011, 7.19943)
+
+ assert(modelWithoutIntercept1.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept1.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ assert(modelWithoutIntercept2.intercept ~== 0 absTol 1E-3)
+ assert(modelWithoutIntercept2.weights ~= weightsWithoutInterceptR relTol 1E-3)
+ }}
}
test("linear regression with intercept with L1 regularization") {
- val trainer1 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- val trainer2 = (new LinearRegression).setElasticNetParam(1.0).setRegParam(0.57)
- .setStandardization(false)
- val model1 = trainer1.fit(dataset)
- val model2 = trainer2.fit(dataset)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.24300
- as.numeric.data.V2. 4.024821
- as.numeric.data.V3. 6.679841
- */
- val interceptR1 = 6.24300
- val weightsR1 = Vectors.dense(4.024821, 6.679841)
-
- assert(model1.intercept ~== interceptR1 relTol 1E-3)
- assert(model1.weights ~= weightsR1 relTol 1E-3)
-
- /*
- weights <- coef(glmnet(features, label, family="gaussian", alpha = 1.0, lambda = 0.57,
- standardize=FALSE))
- > weights
- 3 x 1 sparse Matrix of class "dgCMatrix"
- s0
- (Intercept) 6.416948
- as.numeric.data.V2. 3.893869
- as.numeric.data.V3. 6.724286
- */
- val interceptR2 = 6.416948
- val weightsR2 = Vectors.dense(3.893869, 6.724286)
-
- assert(model2.intercept ~== interceptR2 relTol 1E-3)
- assert(model2.weights ~= weightsR2 relTol 1E-3)
-
-
- model1.transform(dataset).select("features", "prediction").collect().foreach {
- case Row(features: DenseVector, prediction1: Double) =>
- val prediction2 =
- features(0) * model1.weights(0) + features(1) * model1.weights(1) + model1.intercept
- assert(prediction1 ~== prediction2 relTol 1E-5)
- }
+ Seq("auto").foreach { solver => {
--- End diff --
`Seq("auto", "l-bfgs", "normal").foreach { solver =>`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org