You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by mn-mikke <gi...@git.apache.org> on 2018/05/10 22:17:01 UTC
[GitHub] spark pull request #21298: [SPARK-24198][SparkR][SQL] Adding slice function ...
GitHub user mn-mikke opened a pull request:
https://github.com/apache/spark/pull/21298
[SPARK-24198][SparkR][SQL] Adding slice function to SparkR
## What changes were proposed in this pull request?
The PR adds the `slice` function to SparkR. The function returns a subset of consecutive elements from the given array.
```
> df <- createDataFrame(cbind(model = rownames(mtcars), mtcars))
> tmp <- mutate(df, v1 = create_array(df$mpg, df$cyl, df$hp))
> head(select(tmp, slice(tmp$v1, 2L, 2L)))
```
```
slice(v1, 2, 2)
1 6, 110
2 6, 110
3 4, 93
4 6, 110
5 8, 175
6 6, 105
```
## How was this patch tested?
A test added into R/pkg/tests/fulltests/test_sparkSQL.R
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mn-mikke/spark SPARK-24198
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21298.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21298
----
commit fe008dcbeaa996fb48168836bbceedae82662658
Author: Marek Novotny <mn...@...>
Date: 2018-05-10T22:06:36Z
[SPARK-24198][SparkR][SQL] Adding slice function to SparkR
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21298: [SPARK-24198][SparkR][SQL] Adding slice function to Spar...
Posted by mn-mikke <gi...@git.apache.org>.
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/21298
Oh, sorry for that. I'll group several functions next time. Thanks guys.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21298: [SPARK-24198][SparkR][SQL] Adding slice function to Spar...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21298
**[Test build #90539 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90539/testReport)** for PR 21298 at commit [`1c9511e`](https://github.com/apache/spark/commit/1c9511e976f4991185e195bafff388ae7162f24d).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21298: [SPARK-24198][SparkR][SQL] Adding slice function to Spar...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21298
@mn-mikke, let's do this with few more functions together next time per the suggestions from @felixcheung and @shivaram who're PMCs used to R side better than me.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21298: [SPARK-24198][SparkR][SQL] Adding slice function to Spar...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21298
lgtm too except that the nit.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21298: [SPARK-24198][SparkR][SQL] Adding slice function to Spar...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21298
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21298: [SPARK-24198][SparkR][SQL] Adding slice function ...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21298#discussion_r187763906
--- Diff: R/pkg/R/functions.R ---
@@ -3138,6 +3139,23 @@ setMethod("size",
column(jc)
})
+#' @details
+#' \code{slice}: Returns an array containing all the elements in x from the index start
+#' (or starting from the end if start is negative) with the specified length.
+#'
+#' @rdname column_collection_functions
+#' @param start an index indicating the first element occuring in the result.
+#' @param length a number of consecutive elements choosen to the result.
+#'
--- End diff --
nit: remove unneeded line
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21298: [SPARK-24198][SparkR][SQL] Adding slice function to Spar...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21298
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21298: [SPARK-24198][SparkR][SQL] Adding slice function to Spar...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21298
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21298: [SPARK-24198][SparkR][SQL] Adding slice function to Spar...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21298
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90500/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21298: [SPARK-24198][SparkR][SQL] Adding slice function to Spar...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21298
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90539/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21298: [SPARK-24198][SparkR][SQL] Adding slice function to Spar...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21298
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21298: [SPARK-24198][SparkR][SQL] Adding slice function to Spar...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21298
**[Test build #90482 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90482/testReport)** for PR 21298 at commit [`fe008dc`](https://github.com/apache/spark/commit/fe008dcbeaa996fb48168836bbceedae82662658).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21298: [SPARK-24198][SparkR][SQL] Adding slice function to Spar...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21298
Merged to master.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21298: [SPARK-24198][SparkR][SQL] Adding slice function ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21298
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21298: [SPARK-24198][SparkR][SQL] Adding slice function to Spar...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21298
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90482/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21298: [SPARK-24198][SparkR][SQL] Adding slice function to Spar...
Posted by mn-mikke <gi...@git.apache.org>.
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/21298
cc @HyukjinKwon @felixcheung
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21298: [SPARK-24198][SparkR][SQL] Adding slice function to Spar...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21298
**[Test build #90500 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90500/testReport)** for PR 21298 at commit [`8605f13`](https://github.com/apache/spark/commit/8605f13628745ca6b9d62c9704c7b590c8b9a416).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21298: [SPARK-24198][SparkR][SQL] Adding slice function to Spar...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21298
**[Test build #90482 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90482/testReport)** for PR 21298 at commit [`fe008dc`](https://github.com/apache/spark/commit/fe008dcbeaa996fb48168836bbceedae82662658).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21298: [SPARK-24198][SparkR][SQL] Adding slice function to Spar...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21298
**[Test build #90500 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90500/testReport)** for PR 21298 at commit [`8605f13`](https://github.com/apache/spark/commit/8605f13628745ca6b9d62c9704c7b590c8b9a416).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21298: [SPARK-24198][SparkR][SQL] Adding slice function ...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21298#discussion_r187514450
--- Diff: R/pkg/R/functions.R ---
@@ -3124,6 +3125,23 @@ setMethod("size",
column(jc)
})
+#' @details
+#' \code{slice}: Returns an array containing all the elements in x from the index start
+#' (or starting from the end if start is negative) with the specified length.
+#'
+#' @rdname column_collection_functions
+#' @param start an index indicating the first element occuring in the result.
+#' @param length a number of consecutive elements choosen to the result.
+#'
+#' @aliases slice slice,Column-method
+#' @note slice since 2.4.0
+setMethod("slice",
+signature(x = "Column"),
--- End diff --
please fix the ident - see "size" for example
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21298: [SPARK-24198][SparkR][SQL] Adding slice function to Spar...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21298
**[Test build #90539 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90539/testReport)** for PR 21298 at commit [`1c9511e`](https://github.com/apache/spark/commit/1c9511e976f4991185e195bafff388ae7162f24d).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org