You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by MaxGekk <gi...@git.apache.org> on 2018/06/16 04:01:10 UTC

[GitHub] spark pull request #21578: [SPARK-24571][SQL] Support Char literals

GitHub user MaxGekk opened a pull request:

    https://github.com/apache/spark/pull/21578

    [SPARK-24571][SQL] Support Char literals

    ## What changes were proposed in this pull request?
    
    In the PR, I propose to automatically convert a `Literal` with `Char` type to a `Literal` of `String` type. Currently, the following code:
    ```scala
    val df = Seq("Amsterdam", "San Francisco", "London").toDF("city")
    df.where($"city".contains('o')).show(false)
    ```
    fails with the exception:
    ```
    Unsupported literal type class java.lang.Character o
    java.lang.RuntimeException: Unsupported literal type class java.lang.Character o
    at org.apache.spark.sql.catalyst.expressions.Literal$.apply(literals.scala:78)
    ```
    The PR fixes this issue by converting `char` to `string` of length `1`. I believe it makes sense to does not differentiate `char` and `string(1)` in _a unified, multi-language data platform_ like Spark which supports languages like Python.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/MaxGekk/spark-1 support-char-literals

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/21578.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #21578
    
----
commit 97097f58866a492e9bb450de53c5f51ec8ef78e6
Author: Maxim Gekk <ma...@...>
Date:   2018-06-13T22:29:12Z

    A test for converting Char to String

commit 87640c768f9c1c71defb2f476eabbfea67e0f9ed
Author: Maxim Gekk <ma...@...>
Date:   2018-06-13T23:20:39Z

    Support Char in StringConverter

commit 0fb4669f47a5abcfdf255d6fd99f070764343cfc
Author: Maxim Gekk <ma...@...>
Date:   2018-06-13T23:21:43Z

    Evaluate Char literal as String literal

commit 99dfbfe5fc299efe2ece9bea9493dcfe1d5b523d
Author: Maxim Gekk <ma...@...>
Date:   2018-06-14T03:03:56Z

    Added a test for filtering rows by using Char literal

commit 56fd59254d4483b7b9ff866cec7f0cf87c28bb14
Author: Maxim Gekk <ma...@...>
Date:   2018-06-14T03:05:56Z

    Cover the case of java.lang.Character

commit 657f7be117f6b96500c8e2d7d764bf96d94efa8f
Author: Maxim Gekk <ma...@...>
Date:   2018-06-14T03:27:26Z

    Improving of the test

commit 0f09ab2a1ea0584ae61e5f44fd604882208d956c
Author: Maxim Gekk <ma...@...>
Date:   2018-06-16T03:31:05Z

    Merge remote-tracking branch 'origin/master' into char-to-string

commit 42101465583b11665317cba714a778ba5e8404fe
Author: Maxim Gekk <ma...@...>
Date:   2018-06-16T03:47:10Z

    Adding ticket number to test's titles

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21578: [SPARK-24571][SQL] Support Char literals

Posted by MaxGekk <gi...@git.apache.org>.

Github user MaxGekk commented on the issue:

    https://github.com/apache/spark/pull/21578
  
    @rxin Please, review the PR.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21578: [SPARK-24571][SQL] Support Char literals

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21578
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21578: [SPARK-24571][SQL] Support Char literals

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21578
  
    **[Test build #91961 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91961/testReport)** for PR 21578 at commit [`4210146`](https://github.com/apache/spark/commit/42101465583b11665317cba714a778ba5e8404fe).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21578: [SPARK-24571][SQL] Support Char literals

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21578
  
    **[Test build #91974 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91974/testReport)** for PR 21578 at commit [`4210146`](https://github.com/apache/spark/commit/42101465583b11665317cba714a778ba5e8404fe).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21578: [SPARK-24571][SQL] Support Char literals

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21578
  
    **[Test build #92099 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92099/testReport)** for PR 21578 at commit [`450079d`](https://github.com/apache/spark/commit/450079df77f7f0408f70c64315dfdb708022f539).
     * This patch **fails Scala style tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21578: [SPARK-24571][SQL] Support Char literals

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21578
  
    **[Test build #91974 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91974/testReport)** for PR 21578 at commit [`4210146`](https://github.com/apache/spark/commit/42101465583b11665317cba714a778ba5e8404fe).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21578: [SPARK-24571][SQL] Support Char literals

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21578
  
    **[Test build #92100 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92100/testReport)** for PR 21578 at commit [`3926655`](https://github.com/apache/spark/commit/3926655466a75ae61666f27d6434de55e08e967e).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21578: [SPARK-24571][SQL] Support Char literals

Posted by MaxGekk <gi...@git.apache.org>.

Github user MaxGekk commented on the issue:

    https://github.com/apache/spark/pull/21578
  
    jenkins, retest this, please


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21578: [SPARK-24571][SQL] Support Char literals

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21578
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91974/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org