You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by gatorsmile <gi...@git.apache.org> on 2016/08/10 18:05:25 UTC

[GitHub] spark pull request #14584: [SPARK-16999] [SQL] Convert Keyword `path` of Dat...

GitHub user gatorsmile opened a pull request:

    https://github.com/apache/spark/pull/14584

    [SPARK-16999] [SQL] Convert Keyword `path` of Data Source Tables to Lower Case

    ### What changes were proposed in this pull request?
    `path` points to the location of data source tables. The parameters' keywords are case insensitive. However, we do not always follow the rules. Below is the summary of the case sensitivity support:
    
    **Case sensitive**
    - AlterTableSetLocationCommand
    - AlterTableRenameCommand
    - TruncateTableCommand
    - newHadoopConfWithOptions in SessionState
    
    **Case insensitive**
    - ShowCreateTableCommand
    - CreateDataSourceTableCommand
    - CreateDataSourceTableAsSelectCommand
    - getBatch in FileStreamSource
    - write in DataSource
    - inferFileFormatSchema in DataSource
    - sourceSchema in DataSource
    - createSource in DataSource
    - createSink in DataSource
    - resolveRelation in DataSource
    
    It is not maintainable to consider the case sensitivity of `path` in all the current and future implementation. This PR is to convert the user specified keyword name to lower case for `path` and clean the related code. 
    
    ### How was this patch tested?
    Added the related tests.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/gatorsmile/spark alterTableSetLocation

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/14584.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #14584
    
----
commit 314a94f894cb8cd038acca11007e4e1982379274
Author: gatorsmile <ga...@gmail.com>
Date:   2016-08-10T05:10:48Z

    fix.

commit b7c97ae4f05b142c39118ff5b091f5e1520207a6
Author: gatorsmile <ga...@gmail.com>
Date:   2016-08-10T17:49:20Z

    fix.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #14584: [SPARK-16999] [SQL] Convert Keyword `path` of Data Sourc...

Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on the issue:

    https://github.com/apache/spark/pull/14584
  
    I think the `case-sensitive` config is only about column resolution, the data source options should be always case insensitive. What do you think? cc @rxin @yhuai 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #14584: [SPARK-16999] [SQL] Convert Keyword `path` of Data Sourc...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/14584
  
    **[Test build #63550 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63550/consoleFull)** for PR 14584 at commit [`b7c97ae`](https://github.com/apache/spark/commit/b7c97ae4f05b142c39118ff5b091f5e1520207a6).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #14584: [SPARK-16999] [SQL] Convert Keyword `path` of Dat...

Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile closed the pull request at:

    https://github.com/apache/spark/pull/14584


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #14584: [SPARK-16999] [SQL] Convert Keyword `path` of Data Sourc...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/14584
  
    Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #14584: [SPARK-16999] [SQL] Convert Keyword `path` of Data Sourc...

Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:

    https://github.com/apache/spark/pull/14584
  
    @rxin @yhuai @cloud-fan A question about user specified `option`. Based on the description in [code](https://github.com/apache/spark/blob/18c2c92580bdc27aa5129d9e7abda418a3633ea6/sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala#L73-L74)
    
    > Note: the parameters' keywords are case insensitive and this insensitivity is enforced by the Map that is passed to the function.
    
    Should we convert all the option keywords to lower case at the very beginning? It can simplify the code and make all of them case insensitive. We still have the same issue in the other places. For example, in JDBC, `isolationLevel` is also case sensitive.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #14584: [SPARK-16999] [SQL] Convert Keyword `path` of Data Sourc...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/14584
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63550/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #14584: [SPARK-16999] [SQL] Convert Keyword `path` of Data Sourc...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/14584
  
    **[Test build #63550 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63550/consoleFull)** for PR 14584 at commit [`b7c97ae`](https://github.com/apache/spark/commit/b7c97ae4f05b142c39118ff5b091f5e1520207a6).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org