You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by huaxingao <gi...@git.apache.org> on 2018/05/05 00:02:50 UTC

[GitHub] spark pull request #21244: [SPARK-24815]add flatten function to SparkR

GitHub user huaxingao opened a pull request:

    https://github.com/apache/spark/pull/21244

    [SPARK-24815]add flatten function to SparkR

    ## What changes were proposed in this pull request?
    
    add array flatten function to SparkR
    
    ## How was this patch tested?
    
    Unit tests were added in R/pkg/tests/fulltests/test_sparkSQL.R


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/huaxingao/spark spark-24185

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/21244.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #21244
    
----
commit bfccf18b6df0b6a25674001b0fe13c7bc289b806
Author: Huaxin Gao <hu...@...>
Date:   2018-05-04T23:55:20Z

    [SPARK-24815]add flatten function to SparkR

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #21244: [SPARK-24185][SparkR][SQL]add flatten function to...

Posted by HyukjinKwon <gi...@git.apache.org>.

Github user HyukjinKwon commented on a diff in the pull request:

    https://github.com/apache/spark/pull/21244#discussion_r186276677
  
    --- Diff: R/pkg/tests/fulltests/test_sparkSQL.R ---
    @@ -1502,6 +1502,12 @@ test_that("column functions", {
       result <- collect(select(df, sort_array(df[[1]])))[[1]]
       expect_equal(result, list(list(1L, 2L, 3L), list(4L, 5L, 6L)))
     
    +  # Test flattern
    --- End diff --
    
    not a big deal at all but I'd say `flattern()` for consistency. I think detail and consistency are the key .. 


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #21244: [SPARK-24185][SparkR][SQL]add flatten function to...

Posted by HyukjinKwon <gi...@git.apache.org>.

Github user HyukjinKwon commented on a diff in the pull request:

    https://github.com/apache/spark/pull/21244#discussion_r186276706
  
    --- Diff: R/pkg/R/functions.R ---
    @@ -3035,6 +3036,19 @@ setMethod("array_position",
                 column(jc)
               })
     
    +#' @details
    +#' \code{flatten}: Transforms an array of arrays into a single array.
    --- End diff --
    
    not a big deal but let's match the doc too with Python and Scala (not to SQL one) since that's usual so far. Please fix it in the PRs for other functions later.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #21244: [SPARK-24185][SparkR][SQL]add flatten function to...

Posted by huaxingao <gi...@git.apache.org>.

Github user huaxingao commented on a diff in the pull request:

    https://github.com/apache/spark/pull/21244#discussion_r186275756
  
    --- Diff: R/pkg/R/generics.R ---
    @@ -918,6 +918,10 @@ setGeneric("explode_outer", function(x) { standardGeneric("explode_outer") })
     #' @name NULL
     setGeneric("expr", function(x) { standardGeneric("expr") })
     
    +#' @rdname column_collection_functions
    +#' @name NULL
    +setGeneric("flatten", function(x, value) { standardGeneric("flatten") })
    --- End diff --
    
    Thanks for catching the problem. Will correct. 


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21244: [SPARK-24185][SparkR][SQL]add flatten function to SparkR

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21244
  
    **[Test build #90261 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90261/testReport)** for PR 21244 at commit [`fe769df`](https://github.com/apache/spark/commit/fe769df4317ef7c0e1b060a7064cc9d1ad9ed806).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21244: [SPARK-24185][SparkR][SQL]add flatten function to SparkR

Posted by viirya <gi...@git.apache.org>.

Github user viirya commented on the issue:

    https://github.com/apache/spark/pull/21244
  
    cc @HyukjinKwon 


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org