You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by squito <gi...@git.apache.org> on 2018/09/18 14:24:47 UTC

[GitHub] spark pull request #22452: [SPARK-25456][SQL][TEST] Fix PythonForeachWriterS...

GitHub user squito opened a pull request:

    https://github.com/apache/spark/pull/22452

    [SPARK-25456][SQL][TEST] Fix PythonForeachWriterSuite

    PythonForeachWriterSuite was failing because RowQueue now needs to have a handle on a SparkEnv with a SerializerManager, so added a mock env with a serializer manager.
    
    Also fixed a typo in the `finally` that was hiding the real exception.
    
    Tested PythonForeachWriterSuite locally, full tests via jenkins.


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/squito/spark SPARK-25456

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/22452.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #22452
    
----
commit 33725ce4f8632fbb12e9707e71d5cb897e3b6038
Author: Imran Rashid <ir...@...>
Date:   2018-09-18T14:22:57Z

    [SPARK-25456][SQL][TEST] Fix PythonForeachWriterSuite

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request #22452: [SPARK-25456][SQL][TEST] Fix PythonForeachWriterS...

Posted by squito <gi...@git.apache.org>.

Github user squito commented on a diff in the pull request:

    https://github.com/apache/spark/pull/22452#discussion_r218496943
  
    --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/python/PythonForeachWriterSuite.scala ---
    @@ -75,15 +78,20 @@ class PythonForeachWriterSuite extends SparkFunSuite with Eventually {
             tester = new BufferTester(memBytes, sleepPerRowReadMs)
             f(tester)
           } finally {
    -        if (tester == null) tester.close()
    +        if (tester != null) tester.close()
    --- End diff --
    
    yes, but that NPE just covered up another NPE from `SparkEnv.get.serializerManager`


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22452: [SPARK-25456][SQL][TEST] Fix PythonForeachWriterSuite

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22452
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3196/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22452: [SPARK-25456][SQL][TEST] Fix PythonForeachWriterSuite

Posted by squito <gi...@git.apache.org>.

Github user squito commented on the issue:

    https://github.com/apache/spark/pull/22452
  
    merged to master & 2.4


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22452: [SPARK-25456][SQL][TEST] Fix PythonForeachWriterSuite

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22452
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22452: [SPARK-25456][SQL][TEST] Fix PythonForeachWriterSuite

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22452
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22452: [SPARK-25456][SQL][TEST] Fix PythonForeachWriterSuite

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22452
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22452: [SPARK-25456][SQL][TEST] Fix PythonForeachWriterSuite

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/22452
  
    **[Test build #96183 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96183/testReport)** for PR 22452 at commit [`33725ce`](https://github.com/apache/spark/commit/33725ce4f8632fbb12e9707e71d5cb897e3b6038).
     * This patch **fails Spark unit tests**.
     * This patch merges cleanly.
     * This patch adds the following public classes _(experimental)_:
      * `class PythonForeachWriterSuite extends SparkFunSuite with Eventually with MockitoSugar `


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22452: [SPARK-25456][SQL][TEST] Fix PythonForeachWriterSuite

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22452
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96189/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22452: [SPARK-25456][SQL][TEST] Fix PythonForeachWriterSuite

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/22452
  
    **[Test build #96189 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96189/testReport)** for PR 22452 at commit [`33725ce`](https://github.com/apache/spark/commit/33725ce4f8632fbb12e9707e71d5cb897e3b6038).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds the following public classes _(experimental)_:
      * `class PythonForeachWriterSuite extends SparkFunSuite with Eventually with MockitoSugar `


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22452: [SPARK-25456][SQL][TEST] Fix PythonForeachWriterSuite

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/22452
  
    **[Test build #96183 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96183/testReport)** for PR 22452 at commit [`33725ce`](https://github.com/apache/spark/commit/33725ce4f8632fbb12e9707e71d5cb897e3b6038).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org