You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by MaxGekk <gi...@git.apache.org> on 2018/11/02 10:26:32 UTC

[GitHub] spark pull request #22929: [SPARK-25927][SQL] Fix number of partitions retur...

GitHub user MaxGekk opened a pull request:

    https://github.com/apache/spark/pull/22929

    [SPARK-25927][SQL] Fix number of partitions returned by outputPartitioning

    ## What changes were proposed in this pull request?
    
    In the PR, I propose to make the `outputPartitioning()` method abstract and implement it in child nodes to return actual number of partitions. Currently the method returns 0 partitions by default which is wrong in most cases.
    
    ## How was this patch tested?
    
    It was tested by SqlQueryTestSuite.


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/MaxGekk/spark-1 output-partitioning

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/22929.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #22929
    
----
commit c1291848d9443c6ea0dd009a0d107473a30da1a0
Author: Maxim Gekk <ma...@...>
Date:   2018-11-02T08:41:09Z

    Fix number partitions returned by outputPartitioning

commit 25f57f92bfcf28b1b29d75dde75fc2dd358488cb
Author: Maxim Gekk <ma...@...>
Date:   2018-11-02T09:59:54Z

    Fix number of partitions for union

commit 4484b242af756b0afd7f0e947512e72069e175ee
Author: Maxim Gekk <ma...@...>
Date:   2018-11-02T10:14:37Z

    Replacing def by lazy val

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22929: [SPARK-25927][SQL] Fix number of partitions returned by ...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22929
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98393/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22929: [SPARK-25927][SQL] Fix number of partitions returned by ...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/22929
  
    **[Test build #98677 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98677/testReport)** for PR 22929 at commit [`fc765f0`](https://github.com/apache/spark/commit/fc765f021131da0b3291d2f7ba1238db5430ca57).
     * This patch **fails Spark unit tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22929: [SPARK-25927][SQL] Fix number of partitions returned by ...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22929
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98400/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22929: [SPARK-25927][SQL] Fix number of partitions returned by ...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22929
  
    Merged build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22929: [SPARK-25927][SQL] Fix number of partitions returned by ...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/22929
  
    **[Test build #98393 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98393/testReport)** for PR 22929 at commit [`ec5fd9b`](https://github.com/apache/spark/commit/ec5fd9b5a141c6514814c9b1b677739fc15a6f4c).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22929: [SPARK-25927][SQL] Fix number of partitions returned by ...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/22929
  
    **[Test build #98393 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98393/testReport)** for PR 22929 at commit [`ec5fd9b`](https://github.com/apache/spark/commit/ec5fd9b5a141c6514814c9b1b677739fc15a6f4c).
     * This patch **fails Spark unit tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22929: [SPARK-25927][SQL] Fix number of partitions returned by ...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22929
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98677/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22929: [SPARK-25927][SQL] Fix number of partitions returned by ...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22929
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98392/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22929: [SPARK-25927][SQL] Fix number of partitions returned by ...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/22929
  
    **[Test build #98677 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98677/testReport)** for PR 22929 at commit [`fc765f0`](https://github.com/apache/spark/commit/fc765f021131da0b3291d2f7ba1238db5430ca57).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org