You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by MaxGekk <gi...@git.apache.org> on 2018/05/23 17:57:53 UTC

[GitHub] spark pull request #21414: [SPARK-24368][SQL] Removing columnPruning from CS...

GitHub user MaxGekk opened a pull request:

    https://github.com/apache/spark/pull/21414

    [SPARK-24368][SQL] Removing columnPruning from CSVOptions

    ## What changes were proposed in this pull request?
    
    In the PR, I removed the private `columnPruning` value from `CSVOptions`. The flag is initialized _in place_ by reading of `spark.sql.csv.parser.columnPruning.enabled`. It should make `UnivocityParserSuite` more stable.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/MaxGekk/spark-1 univocity-parser-suite

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/21414.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #21414
    
----
commit 6fdd43585abcefcfb911039910ce7dd2a55df4e5
Author: Maxim Gekk <ma...@...>
Date:   2018-05-23T17:52:49Z

    Removing columnPruning from CSVOptions

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21414: [SPARK-24368][SQL] Removing columnPruning from CSVOption...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21414
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91057/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21414: [SPARK-24368][SQL] Removing columnPruning from CSVOption...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21414
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21414: [SPARK-24368][SQL] Removing columnPruning from CSVOption...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/21414
  
    Can one of the admins verify this patch?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #21414: [SPARK-24368][SQL] Removing columnPruning from CS...

Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on a diff in the pull request:

    https://github.com/apache/spark/pull/21414#discussion_r190354077
  
    --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVOptions.scala ---
    @@ -25,7 +25,6 @@ import org.apache.commons.lang3.time.FastDateFormat
     
     import org.apache.spark.internal.Logging
     import org.apache.spark.sql.catalyst.util._
    -import org.apache.spark.sql.internal.SQLConf
     
     class CSVOptions(
         @transient val parameters: CaseInsensitiveMap[String],
    --- End diff --
    
    We can add the columnPruning conf here, like what we are doing for  `sparkSession.sessionState.conf.sessionLocalTimeZone`


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21414: [SPARK-24368][SQL] Removing columnPruning from CSVOption...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21414
  
    **[Test build #91057 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91057/testReport)** for PR 21414 at commit [`6fdd435`](https://github.com/apache/spark/commit/6fdd43585abcefcfb911039910ce7dd2a55df4e5).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #21414: [SPARK-24368][SQL] Removing columnPruning from CS...

Posted by MaxGekk <gi...@git.apache.org>.
Github user MaxGekk closed the pull request at:

    https://github.com/apache/spark/pull/21414


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #21414: [SPARK-24368][SQL] Removing columnPruning from CSVOption...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/21414
  
    **[Test build #91057 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91057/testReport)** for PR 21414 at commit [`6fdd435`](https://github.com/apache/spark/commit/6fdd43585abcefcfb911039910ce7dd2a55df4e5).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org