You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by fe...@apache.org on 2017/04/07 19:29:48 UTC
spark git commit: [SPARK-20258][DOC][SPARKR] Fix SparkR logistic
regression example in programming guide (did not converge)
Repository: spark
Updated Branches:
refs/heads/master 8feb799af -> 1ad73f0a2
[SPARK-20258][DOC][SPARKR] Fix SparkR logistic regression example in programming guide (did not converge)
## What changes were proposed in this pull request?
SparkR logistic regression example did not converge in programming guide (for IRWLS). All estimates are essentially zero:
```
training2 <- read.df("data/mllib/sample_binary_classification_data.txt", source = "libsvm")
df_list2 <- randomSplit(training2, c(7,3), 2)
binomialDF <- df_list2[[1]]
binomialTestDF <- df_list2[[2]]
binomialGLM <- spark.glm(binomialDF, label ~ features, family = "binomial")
17/04/07 11:42:03 WARN WeightedLeastSquares: Cholesky solver failed due to singular covariance matrix. Retrying with Quasi-Newton solver.
> summary(binomialGLM)
Coefficients:
Estimate
(Intercept) 9.0255e+00
features_0 0.0000e+00
features_1 0.0000e+00
features_2 0.0000e+00
features_3 0.0000e+00
features_4 0.0000e+00
features_5 0.0000e+00
features_6 0.0000e+00
features_7 0.0000e+00
```
Author: actuaryzhang <ac...@gmail.com>
Closes #17571 from actuaryzhang/programGuide2.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/1ad73f0a
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/1ad73f0a
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/1ad73f0a
Branch: refs/heads/master
Commit: 1ad73f0a21d8007d8466ef8756f751c0ab6a9d1f
Parents: 8feb799
Author: actuaryzhang <ac...@gmail.com>
Authored: Fri Apr 7 12:29:45 2017 -0700
Committer: Felix Cheung <fe...@apache.org>
Committed: Fri Apr 7 12:29:45 2017 -0700
----------------------------------------------------------------------
examples/src/main/r/ml/glm.R | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/1ad73f0a/examples/src/main/r/ml/glm.R
----------------------------------------------------------------------
diff --git a/examples/src/main/r/ml/glm.R b/examples/src/main/r/ml/glm.R
index 23141b5..68787f9 100644
--- a/examples/src/main/r/ml/glm.R
+++ b/examples/src/main/r/ml/glm.R
@@ -27,7 +27,7 @@ sparkR.session(appName = "SparkR-ML-glm-example")
# $example on$
training <- read.df("data/mllib/sample_multiclass_classification_data.txt", source = "libsvm")
# Fit a generalized linear model of family "gaussian" with spark.glm
-df_list <- randomSplit(training, c(7,3), 2)
+df_list <- randomSplit(training, c(7, 3), 2)
gaussianDF <- df_list[[1]]
gaussianTestDF <- df_list[[2]]
gaussianGLM <- spark.glm(gaussianDF, label ~ features, family = "gaussian")
@@ -44,8 +44,9 @@ gaussianGLM2 <- glm(label ~ features, gaussianDF, family = "gaussian")
summary(gaussianGLM2)
# Fit a generalized linear model of family "binomial" with spark.glm
-training2 <- read.df("data/mllib/sample_binary_classification_data.txt", source = "libsvm")
-df_list2 <- randomSplit(training2, c(7,3), 2)
+training2 <- read.df("data/mllib/sample_multiclass_classification_data.txt", source = "libsvm")
+training2 <- transform(training2, label = cast(training2$label > 1, "integer"))
+df_list2 <- randomSplit(training2, c(7, 3), 2)
binomialDF <- df_list2[[1]]
binomialTestDF <- df_list2[[2]]
binomialGLM <- spark.glm(binomialDF, label ~ features, family = "binomial")
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org