You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by cloud-fan <gi...@git.apache.org> on 2018/09/11 02:28:53 UTC

[GitHub] spark pull request #22388: Revert [SPARK-24882][SQL] improve data source v2 ...

GitHub user cloud-fan opened a pull request:

    https://github.com/apache/spark/pull/22388

    Revert [SPARK-24882][SQL] improve data source v2 API from branch 2.4

    ## What changes were proposed in this pull request?
    
    As discussed in the dev list, we don't want to include this change in Spark 2.4, as it needs data source v2 users to change the implementation intensitively, while they need to change again in next release.
    
    ## How was this patch tested?
    
    existing tests


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/cloud-fan/spark revert

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/22388.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #22388
    
----
commit b4cf7701146675c682d51a72279c9c98a62e21c9
Author: Wenchen Fan <we...@...>
Date:   2018-09-11T00:59:16Z

    Revert "[SPARK-24882][SQL] improve data source v2 API"
    
    This reverts commit e754887182304ad0d622754e33192ebcdd515965.

commit e31ecfa574393971586fa04d93766343f7661399
Author: Wenchen Fan <we...@...>
Date:   2018-09-11T02:24:36Z

    fix import

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95968/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    **[Test build #95968 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95968/testReport)** for PR 22388 at commit [`031ad29`](https://github.com/apache/spark/commit/031ad29305326145510b8065f49ae51109e18653).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    Merged build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by cloud-fan <gi...@git.apache.org>.

Github user cloud-fan commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    cc @rxin @rdblue 


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    **[Test build #95912 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95912/testReport)** for PR 22388 at commit [`e31ecfa`](https://github.com/apache/spark/commit/e31ecfa574393971586fa04d93766343f7661399).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    **[Test build #95968 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95968/testReport)** for PR 22388 at commit [`031ad29`](https://github.com/apache/spark/commit/031ad29305326145510b8065f49ae51109e18653).
     * This patch **fails to build**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    **[Test build #95967 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95967/testReport)** for PR 22388 at commit [`234b67a`](https://github.com/apache/spark/commit/234b67a99f2575ec14fc395e8b1a44cc018721c4).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by jose-torres <gi...@git.apache.org>.

Github user jose-torres commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    MicroBatchExecution.scala and ContinuousExecution.scala look right after the revert, although it would be helpful to understand what the diff is between this and a straight `git revert`.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by kiszk <gi...@git.apache.org>.

Github user kiszk commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    Incorrect import still may exist there.
    ```
    [error] /home/jenkins/workspace/SparkPullRequestBuilder/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousTextSocketSource.scala:40: object SupportsDeprecatedScanRow is not a member of package org.apache.spark.sql.sources.v2.reader
    [error] import org.apache.spark.sql.sources.v2.reader.{InputPartition, InputPartitionReader, SupportsDeprecatedScanRow}
    [error]        ^
    ```


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by dongjoon-hyun <gi...@git.apache.org>.

Github user dongjoon-hyun commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    This seems to be the complete revert of SPARK-24882 (#22009 and the followup #22245). Am I right?



---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    **[Test build #95912 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95912/testReport)** for PR 22388 at commit [`e31ecfa`](https://github.com/apache/spark/commit/e31ecfa574393971586fa04d93766343f7661399).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by rdblue <gi...@git.apache.org>.

Github user rdblue commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    Thanks for doing this, @cloud-fan! Sorry I'm late to reply, I was at Strata all last week.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by cloud-fan <gi...@git.apache.org>.

Github user cloud-fan commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    I can't recall the exact conflicts. There are only 2 commits touched these 2 files after my PR, and I carefully checked and theese changs are still there.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    **[Test build #95986 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95986/testReport)** for PR 22388 at commit [`be13162`](https://github.com/apache/spark/commit/be13162228dea05f17fe200751bca22e23f743b9).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by cloud-fan <gi...@git.apache.org>.

Github user cloud-fan commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    As we discussed in the dev list, we only want to revert it from 2.4. I'm closing it now.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    **[Test build #95986 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95986/testReport)** for PR 22388 at commit [`be13162`](https://github.com/apache/spark/commit/be13162228dea05f17fe200751bca22e23f743b9).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2995/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95986/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by gatorsmile <gi...@git.apache.org>.

Github user gatorsmile commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    Unable to merge to master. @cloud-fan Could you submit another PR to the master branch?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3028/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95967/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    Merged build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22388
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95912/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org