You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2018/01/01 22:00:33 UTC
spark git commit: [SPARK-21616][SPARKR][DOCS] update R migration
guide and vignettes
Repository: spark
Updated Branches:
refs/heads/master f5b7714e0 -> 7a702d8d5
[SPARK-21616][SPARKR][DOCS] update R migration guide and vignettes
## What changes were proposed in this pull request?
update R migration guide and vignettes
## How was this patch tested?
manually
Author: Felix Cheung <fe...@hotmail.com>
Closes #20106 from felixcheung/rreleasenote23.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/7a702d8d
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/7a702d8d
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/7a702d8d
Branch: refs/heads/master
Commit: 7a702d8d5ed830de5d2237f136b08bd18deae037
Parents: f5b7714
Author: Felix Cheung <fe...@hotmail.com>
Authored: Tue Jan 2 07:00:31 2018 +0900
Committer: hyukjinkwon <gu...@gmail.com>
Committed: Tue Jan 2 07:00:31 2018 +0900
----------------------------------------------------------------------
R/pkg/tests/fulltests/test_Windows.R | 1 +
R/pkg/vignettes/sparkr-vignettes.Rmd | 3 +--
docs/sparkr.md | 6 ++++++
3 files changed, 8 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/7a702d8d/R/pkg/tests/fulltests/test_Windows.R
----------------------------------------------------------------------
diff --git a/R/pkg/tests/fulltests/test_Windows.R b/R/pkg/tests/fulltests/test_Windows.R
index b2ec6c6..209827d 100644
--- a/R/pkg/tests/fulltests/test_Windows.R
+++ b/R/pkg/tests/fulltests/test_Windows.R
@@ -14,6 +14,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
+
context("Windows-specific tests")
test_that("sparkJars tag in SparkContext", {
http://git-wip-us.apache.org/repos/asf/spark/blob/7a702d8d/R/pkg/vignettes/sparkr-vignettes.Rmd
----------------------------------------------------------------------
diff --git a/R/pkg/vignettes/sparkr-vignettes.Rmd b/R/pkg/vignettes/sparkr-vignettes.Rmd
index 8c4ea2f..2e66242 100644
--- a/R/pkg/vignettes/sparkr-vignettes.Rmd
+++ b/R/pkg/vignettes/sparkr-vignettes.Rmd
@@ -391,8 +391,7 @@ We convert `mpg` to `kmpg` (kilometers per gallon). `carsSubDF` is a `SparkDataF
```{r}
carsSubDF <- select(carsDF, "model", "mpg")
-schema <- structType(structField("model", "string"), structField("mpg", "double"),
- structField("kmpg", "double"))
+schema <- "model STRING, mpg DOUBLE, kmpg DOUBLE"
out <- dapply(carsSubDF, function(x) { x <- cbind(x, x$mpg * 1.61) }, schema)
head(collect(out))
```
http://git-wip-us.apache.org/repos/asf/spark/blob/7a702d8d/docs/sparkr.md
----------------------------------------------------------------------
diff --git a/docs/sparkr.md b/docs/sparkr.md
index a3254e7..997ea60 100644
--- a/docs/sparkr.md
+++ b/docs/sparkr.md
@@ -657,3 +657,9 @@ You can inspect the search path in R with [`search()`](https://stat.ethz.ch/R-ma
- By default, derby.log is now saved to `tempdir()`. This will be created when instantiating the SparkSession with `enableHiveSupport` set to `TRUE`.
- `spark.lda` was not setting the optimizer correctly. It has been corrected.
- Several model summary outputs are updated to have `coefficients` as `matrix`. This includes `spark.logit`, `spark.kmeans`, `spark.glm`. Model summary outputs for `spark.gaussianMixture` have added log-likelihood as `loglik`.
+
+## Upgrading to SparkR 2.3.0
+
+ - The `stringsAsFactors` parameter was previously ignored with `collect`, for example, in `collect(createDataFrame(iris), stringsAsFactors = TRUE))`. It has been corrected.
+ - For `summary`, option for statistics to compute has been added. Its output is changed from that from `describe`.
+ - A warning can be raised if versions of SparkR package and the Spark JVM do not match.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org