You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by lw-lin <gi...@git.apache.org> on 2016/06/05 11:02:45 UTC

[GitHub] spark pull request #13518: [SPARK-15472][SQL] Add support for writing in `cs...

GitHub user lw-lin opened a pull request:

    https://github.com/apache/spark/pull/13518

    [SPARK-15472][SQL] Add support for writing in `csv`, `json`, `text` formats in Structured Streaming

    ## What changes were proposed in this pull request?
    
    This patch adds support for writing in `csv`, `json`, `text` formats in Structured Streaming:
    
    **1. at a high level, this patch forms the following hierarchy**(`text` as an example):
    ```
                    <OutputWriter>
                          ↑
                 TextOutputWriterBase
                     ↗          ↖
    BatchTextOutputWriter   StreamingTextOutputWriter
    ```
    ```
                        <OutputWriterFactory>
                            ↗          ↖
    BatchTextOutputWriterFactory   StreamingOutputWriterFactory
                                                  ↑
                                  StreamingTextOutputWriterFactory
    ```
    The `StreamingTextOutputWriter` and other 'streaming' output writers would write data **without** using an `OutputCommitter`. This was the same approach taken by [SPARK-14716](https://github.com/apache/spark/pull/12409).
    
    **2. to support compression, this patch attaches an extension to the path assigned by `FileStreamSink`.** For example, if we are writing out using the `gzip` compression and `FileStreamSink` assigns path `{$uuid}` to a text writer, then in the end the file written out will be `${uuid}.txt.gz` -- so that when we read the file back, we'll correctly interpret it as `gzip` compressed.
    
    
     ## How was this patch tested?
    
    `FileStreamSinkSuite` is much more expanded to cover the added `csv`, `json`, `text` format:
    
    ```scala
    test(" csv - unpartitioned data - codecs: none/gzip")
    test("json - unpartitioned data - codecs: none/gzip")
    test("text - unpartitioned data - codecs: none/gzip")
    
    test(" csv - partitioned data - codecs: none/gzip")
    test("json - partitioned data - codecs: none/gzip")
    test("text - partitioned data - codecs: none/gzip")
    
    test(" csv - unpartitioned writing and batch reading - codecs: none/gzip")
    test("json - unpartitioned writing and batch reading - codecs: none/gzip")
    test("text - unpartitioned writing and batch reading - codecs: none/gzip")
    
    test(" csv - partitioned writing and batch reading - codecs: none/gzip")
    test("json - partitioned writing and batch reading - codecs: none/gzip")
    test("text - partitioned writing and batch reading - codecs: none/gzip")
    ```

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/lw-lin/spark add-csv-json-text-for-ss

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/13518.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #13518
    
----
commit 97034f9aeb092b10e1606e60a8e6b4878ebd54cf
Author: Liwei Lin <lw...@gmail.com>
Date:   2016-06-05T09:03:04Z

    Add csv, json, text

commit 2035b597b44aa519d8da3b155036446f88b3050e
Author: Liwei Lin <lw...@gmail.com>
Date:   2016-06-05T09:03:15Z

    Fix parquet extension

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    **[Test build #60219 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60219/consoleFull)** for PR 13518 at commit [`bc28f41`](https://github.com/apache/spark/commit/bc28f4112ca9eca6a9f1602a891dd0388fa3185c).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [SPARK-15472][SQL] Add support for writing in `csv`, `js...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    **[Test build #60021 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60021/consoleFull)** for PR 13518 at commit [`4737361`](https://github.com/apache/spark/commit/4737361489fd680405b291ec498ab91374685ffe).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60103/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #13518: [WIP][SPARK-15472][SQL] Add support for writing i...

Posted by lw-lin <gi...@git.apache.org>.

GitHub user lw-lin reopened a pull request:

    https://github.com/apache/spark/pull/13518

    [WIP][SPARK-15472][SQL] Add support for writing in `csv`, `json`, `text` formats in Structured Streaming

    ## What changes were proposed in this pull request?
    
    This patch adds support for writing in `csv`, `json`, `text` formats in Structured Streaming:
    
    **1. at a high level, this patch forms the following hierarchy**(`text` as an example):
    ```
                    <OutputWriter>
                          ↑
                 TextOutputWriterBase
                     ↗          ↖
    BatchTextOutputWriter   StreamingTextOutputWriter
    ```
    ```
                        <OutputWriterFactory>
                            ↗          ↖
    BatchTextOutputWriterFactory   StreamingOutputWriterFactory
                                                  ↑
                                  StreamingTextOutputWriterFactory
    ```
    The `StreamingTextOutputWriter` and other 'streaming' output writers would write data **without** using an `OutputCommitter`. This was the same approach taken by [SPARK-14716](https://github.com/apache/spark/pull/12409).
    
    **2. to support compression, this patch attaches an extension to the path assigned by `FileStreamSink`**, which is slightly different from [SPARK-14716](https://github.com/apache/spark/pull/12409). For example, if we are writing out using the `gzip` compression and `FileStreamSink` assigns path `${uuid}` to a text writer, then in the end the file written out will be `${uuid}.txt.gz` -- so that when we read the file back, we'll correctly interpret it as `gzip` compressed.
    
    ## How was this patch tested?
    
    `FileStreamSinkSuite` is expanded much more to cover the added `csv`, `json`, `text` formats:
    
    ```scala
    test(" csv - unpartitioned data - codecs: none/gzip")
    test("json - unpartitioned data - codecs: none/gzip")
    test("text - unpartitioned data - codecs: none/gzip")
    
    test(" csv - partitioned data - codecs: none/gzip")
    test("json - partitioned data - codecs: none/gzip")
    test("text - partitioned data - codecs: none/gzip")
    
    test(" csv - unpartitioned writing and batch reading - codecs: none/gzip")
    test("json - unpartitioned writing and batch reading - codecs: none/gzip")
    test("text - unpartitioned writing and batch reading - codecs: none/gzip")
    
    test(" csv - partitioned writing and batch reading - codecs: none/gzip")
    test("json - partitioned writing and batch reading - codecs: none/gzip")
    test("text - partitioned writing and batch reading - codecs: none/gzip")
    ```

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/lw-lin/spark add-csv-json-text-for-ss

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/13518.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #13518
    
----
commit c70083e9f76c20f6bf48e7ec821452f9bf63783a
Author: Liwei Lin <lw...@gmail.com>
Date:   2016-06-05T09:03:04Z

    Add csv, json, text

commit bc28f4112ca9eca6a9f1602a891dd0388fa3185c
Author: Liwei Lin <lw...@gmail.com>
Date:   2016-06-09T03:31:59Z

    Fix parquet extension

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60108/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [SPARK-15472][SQL] Add support for writing in `csv`, `js...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    **[Test build #60031 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60031/consoleFull)** for PR 13518 at commit [`90d02c4`](https://github.com/apache/spark/commit/90d02c4a10c14af83bbed985e36ef99a1edaa48b).
     * This patch **fails Spark unit tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60219/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60104/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60640/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [SPARK-15472][SQL] Add support for writing in `csv`, `js...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60113/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    **[Test build #60114 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60114/consoleFull)** for PR 13518 at commit [`c5ac63e`](https://github.com/apache/spark/commit/c5ac63e139c37e9e29c071ef17bfd0c45b40676b).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    **[Test build #60113 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60113/consoleFull)** for PR 13518 at commit [`aa61f1d`](https://github.com/apache/spark/commit/aa61f1dab4fa367296fa83948c173985a289a629).
     * This patch **fails executing the `dev/run-tests` script**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60041/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    **[Test build #60033 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60033/consoleFull)** for PR 13518 at commit [`43b68d4`](https://github.com/apache/spark/commit/43b68d426e9b64061095eca7a1db0e762843adef).
     * This patch **fails Spark unit tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60218/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [SPARK-15472][SQL] Add support for writing in `csv`, `js...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    **[Test build #60009 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60009/consoleFull)** for PR 13518 at commit [`4737361`](https://github.com/apache/spark/commit/4737361489fd680405b291ec498ab91374685ffe).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60039/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [SPARK-15472][SQL] Add support for writing in `csv`, `js...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    **[Test build #60008 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60008/consoleFull)** for PR 13518 at commit [`2035b59`](https://github.com/apache/spark/commit/2035b597b44aa519d8da3b155036446f88b3050e).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    **[Test build #60104 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60104/consoleFull)** for PR 13518 at commit [`b64afc6`](https://github.com/apache/spark/commit/b64afc64d3121479eb5c3f8c8b5663b6e05349b7).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    **[Test build #60640 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60640/consoleFull)** for PR 13518 at commit [`ec51f98`](https://github.com/apache/spark/commit/ec51f9878d971a14b56375ae4f8d6f33ec64e751).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by lw-lin <gi...@git.apache.org>.

Github user lw-lin commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Jenkins retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    **[Test build #60114 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60114/consoleFull)** for PR 13518 at commit [`c5ac63e`](https://github.com/apache/spark/commit/c5ac63e139c37e9e29c071ef17bfd0c45b40676b).
     * This patch **fails executing the `dev/run-tests` script**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [SPARK-15472][SQL] Add support for writing in `csv`, `js...

Posted by lw-lin <gi...@git.apache.org>.

Github user lw-lin commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Jenkins retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60114/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    **[Test build #60043 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60043/consoleFull)** for PR 13518 at commit [`eb2090c`](https://github.com/apache/spark/commit/eb2090ce9fa04efbd23370f7d3e6cb98fd0b4c74).
     * This patch **fails executing the `dev/run-tests` script**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    **[Test build #60115 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60115/consoleFull)** for PR 13518 at commit [`cc84273`](https://github.com/apache/spark/commit/cc842733d7ad2644ed8ddb7ebd9dcc4def301130).
     * This patch **fails executing the `dev/run-tests` script**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [SPARK-15472][SQL] Add support for writing in `csv`, `js...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    **[Test build #60031 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60031/consoleFull)** for PR 13518 at commit [`90d02c4`](https://github.com/apache/spark/commit/90d02c4a10c14af83bbed985e36ef99a1edaa48b).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60101/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    **[Test build #60101 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60101/consoleFull)** for PR 13518 at commit [`1e9af1c`](https://github.com/apache/spark/commit/1e9af1cef892706de8b07728c192dd8ca5e5851e).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #13518: [WIP][SPARK-15472][SQL] Add support for writing i...

Posted by lw-lin <gi...@git.apache.org>.

GitHub user lw-lin reopened a pull request:

    https://github.com/apache/spark/pull/13518

    [WIP][SPARK-15472][SQL] Add support for writing in `csv`, `json`, `text` formats in Structured Streaming

    ## What changes were proposed in this pull request?
    
    This patch adds support for writing in `csv`, `json`, `text` formats in Structured Streaming:
    
    **1. at a high level, this patch forms the following hierarchy**(`text` as an example):
    ```
                    <OutputWriter>
                          ↑
                 TextOutputWriterBase
                     ↗          ↖
    BatchTextOutputWriter   StreamingTextOutputWriter
    ```
    ```
                        <OutputWriterFactory>
                            ↗          ↖
    BatchTextOutputWriterFactory   StreamingOutputWriterFactory
                                                  ↑
                                  StreamingTextOutputWriterFactory
    ```
    The `StreamingTextOutputWriter` and other 'streaming' output writers would write data **without** using an `OutputCommitter`. This was the same approach taken by [SPARK-14716](https://github.com/apache/spark/pull/12409).
    
    **2. to support compression, this patch attaches an extension to the path assigned by `FileStreamSink`**, which is slightly different from [SPARK-14716](https://github.com/apache/spark/pull/12409). For example, if we are writing out using the `gzip` compression and `FileStreamSink` assigns path `${uuid}` to a text writer, then in the end the file written out will be `${uuid}.txt.gz` -- so that when we read the file back, we'll correctly interpret it as `gzip` compressed.
    
    ## How was this patch tested?
    
    `FileStreamSinkSuite` is expanded much more to cover the added `csv`, `json`, `text` formats:
    
    ```scala
    test(" csv - unpartitioned data - codecs: none/gzip")
    test("json - unpartitioned data - codecs: none/gzip")
    test("text - unpartitioned data - codecs: none/gzip")
    
    test(" csv - partitioned data - codecs: none/gzip")
    test("json - partitioned data - codecs: none/gzip")
    test("text - partitioned data - codecs: none/gzip")
    
    test(" csv - unpartitioned writing and batch reading - codecs: none/gzip")
    test("json - unpartitioned writing and batch reading - codecs: none/gzip")
    test("text - unpartitioned writing and batch reading - codecs: none/gzip")
    
    test(" csv - partitioned writing and batch reading - codecs: none/gzip")
    test("json - partitioned writing and batch reading - codecs: none/gzip")
    test("text - partitioned writing and batch reading - codecs: none/gzip")
    ```

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/lw-lin/spark add-csv-json-text-for-ss

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/13518.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #13518
    
----
commit 97034f9aeb092b10e1606e60a8e6b4878ebd54cf
Author: Liwei Lin <lw...@gmail.com>
Date:   2016-06-05T09:03:04Z

    Add csv, json, text

commit 2035b597b44aa519d8da3b155036446f88b3050e
Author: Liwei Lin <lw...@gmail.com>
Date:   2016-06-05T09:03:15Z

    Fix parquet extension

commit 4737361489fd680405b291ec498ab91374685ffe
Author: Liwei Lin <lw...@gmail.com>
Date:   2016-06-05T11:52:14Z

    Fix style

commit 90d02c4a10c14af83bbed985e36ef99a1edaa48b
Author: Liwei Lin <lw...@gmail.com>
Date:   2016-06-06T08:02:32Z

    Fix tests

commit daec480bd16ed52137d32f332debe3806953f4d2
Author: Liwei Lin <lw...@gmail.com>
Date:   2016-06-06T09:03:08Z

    Revert "Fix tests"
    
    This reverts commit 90d02c4a10c14af83bbed985e36ef99a1edaa48b.

commit 43b68d426e9b64061095eca7a1db0e762843adef
Author: Liwei Lin <lw...@gmail.com>
Date:   2016-06-06T09:09:10Z

    Fix tests

commit 56dbb9b4f0f7e2bf76935e0d1d2fc6c6cdf141ff
Author: Liwei Lin <lw...@gmail.com>
Date:   2016-06-06T12:34:07Z

    Investigate test

commit 91e51aed5caf663d5068057e5cf28f21eb768310
Author: Liwei Lin <lw...@gmail.com>
Date:   2016-06-06T12:43:39Z

    Investigate test

commit eb2090ce9fa04efbd23370f7d3e6cb98fd0b4c74
Author: Liwei Lin <lw...@gmail.com>
Date:   2016-06-06T13:00:26Z

    Update run-tests

commit 1e9af1cef892706de8b07728c192dd8ca5e5851e
Author: Liwei Lin <lw...@gmail.com>
Date:   2016-06-07T04:10:34Z

    Investigate test

commit f76b1d9fe5e4970ad66dfb6d4b2fed939b68728c
Author: Liwei Lin <lw...@gmail.com>
Date:   2016-06-07T04:49:12Z

    Fix tests

commit 7fca579d9fec2589f13403b0eb0f2d5f5e6bd52a
Author: Liwei Lin <lw...@gmail.com>
Date:   2016-06-07T04:50:24Z

    Fix tests

commit 2ead307d01d8f908951fbc059b8e49bbc77947b1
Author: Liwei Lin <lw...@gmail.com>
Date:   2016-06-07T05:02:55Z

    Fix tests

commit b64afc64d3121479eb5c3f8c8b5663b6e05349b7
Author: Liwei Lin <lw...@gmail.com>
Date:   2016-06-07T05:35:34Z

    Fix tests

commit 2c52c2c6ac961d0b71c2dd3802cd53d51ba1926a
Author: Liwei Lin <lw...@gmail.com>
Date:   2016-06-07T05:58:58Z

    Fix style

commit aa61f1dab4fa367296fa83948c173985a289a629
Author: Liwei Lin <lw...@gmail.com>
Date:   2016-06-07T06:39:22Z

    Fix tests

commit c5ac63e139c37e9e29c071ef17bfd0c45b40676b
Author: Liwei Lin <lw...@gmail.com>
Date:   2016-06-07T06:53:44Z

    Fix tests

commit cc842733d7ad2644ed8ddb7ebd9dcc4def301130
Author: Liwei Lin <lw...@gmail.com>
Date:   2016-06-07T07:12:46Z

    Investigate tests

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60033/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [SPARK-15472][SQL] Add support for writing in `csv`, `js...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60009/
    Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    **[Test build #60108 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60108/consoleFull)** for PR 13518 at commit [`2c52c2c`](https://github.com/apache/spark/commit/2c52c2c6ac961d0b71c2dd3802cd53d51ba1926a).
     * This patch **fails executing the `dev/run-tests` script**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    **[Test build #60104 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60104/consoleFull)** for PR 13518 at commit [`b64afc6`](https://github.com/apache/spark/commit/b64afc64d3121479eb5c3f8c8b5663b6e05349b7).
     * This patch **fails executing the `dev/run-tests` script**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #13518: [WIP][SPARK-15472][SQL] Add support for writing in `csv`...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/13518
  
    **[Test build #60638 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60638/consoleFull)** for PR 13518 at commit [`ec51f98`](https://github.com/apache/spark/commit/ec51f9878d971a14b56375ae4f8d6f33ec64e751).
     * This patch **fails Spark unit tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org