You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sh...@apache.org on 2016/05/08 07:17:40 UTC
spark git commit: [SPARK-12479][SPARKR] sparkR collect on GroupedData
throws R error "missing value where TRUE/FALSE needed"
Repository: spark
Updated Branches:
refs/heads/master 6e268b9ee -> 454ba4d67
[SPARK-12479][SPARKR] sparkR collect on GroupedData throws R error "missing value where TRUE/FALSE needed"
## What changes were proposed in this pull request?
This PR is a workaround for NA handling in hash code computation.
This PR is on behalf of paulomagalhaes whose PR is https://github.com/apache/spark/pull/10436
## How was this patch tested?
SparkR unit tests.
Author: Sun Rui <su...@gmail.com>
Author: ray <ra...@rays-MacBook-Air.local>
Closes #12976 from sun-rui/SPARK-12479.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/454ba4d6
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/454ba4d6
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/454ba4d6
Branch: refs/heads/master
Commit: 454ba4d67e782369627dfe60261e6648a27b91a0
Parents: 6e268b9
Author: Sun Rui <su...@gmail.com>
Authored: Sun May 8 00:17:36 2016 -0700
Committer: Shivaram Venkataraman <sh...@cs.berkeley.edu>
Committed: Sun May 8 00:17:36 2016 -0700
----------------------------------------------------------------------
R/pkg/R/utils.R | 3 +++
R/pkg/inst/tests/testthat/test_utils.R | 4 ++++
2 files changed, 7 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/454ba4d6/R/pkg/R/utils.R
----------------------------------------------------------------------
diff --git a/R/pkg/R/utils.R b/R/pkg/R/utils.R
index bf67e23..784f737 100644
--- a/R/pkg/R/utils.R
+++ b/R/pkg/R/utils.R
@@ -157,8 +157,11 @@ wrapInt <- function(value) {
# Multiply `val` by 31 and add `addVal` to the result. Ensures that
# integer-overflows are handled at every step.
+#
+# TODO: this function does not handle integer overflow well
mult31AndAdd <- function(val, addVal) {
vec <- c(bitwShiftL(val, c(4, 3, 2, 1, 0)), addVal)
+ vec[is.na(vec)] <- 0
Reduce(function(a, b) {
wrapInt(as.numeric(a) + as.numeric(b))
},
http://git-wip-us.apache.org/repos/asf/spark/blob/454ba4d6/R/pkg/inst/tests/testthat/test_utils.R
----------------------------------------------------------------------
diff --git a/R/pkg/inst/tests/testthat/test_utils.R b/R/pkg/inst/tests/testthat/test_utils.R
index 01694ab..54d2eca 100644
--- a/R/pkg/inst/tests/testthat/test_utils.R
+++ b/R/pkg/inst/tests/testthat/test_utils.R
@@ -164,3 +164,7 @@ test_that("convertToJSaveMode", {
expect_error(convertToJSaveMode("foo"),
'mode should be one of "append", "overwrite", "error", "ignore"') #nolint
})
+
+test_that("hashCode", {
+ expect_error(hashCode("bc53d3605e8a5b7de1e8e271c2317645"), NA)
+})
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org