You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by heary-cao <gi...@git.apache.org> on 2017/05/05 05:49:51 UTC

[GitHub] spark pull request #17868: [CORE]Add new unit tests to ShuffleSuite

GitHub user heary-cao opened a pull request:

    https://github.com/apache/spark/pull/17868

    [CORE]Add new unit tests to ShuffleSuite

    ## What changes were proposed in this pull request?
    
    This PR update to two:
    1.adds the new unit tests.
      testing would be performed when there is no shuffle stage, 
      shuffle will not generate the data file and the index files.
    2.Modify the '[SPARK-4085] rerun map stage if reduce stage cannot find its local shuffle file' unit test, 
      parallelize is 1 but not is 2, Check the index file and delete.
    
    ## How was this patch tested?
    The new unit test.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/heary-cao/spark ShuffleSuite

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/17868.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #17868
    
----
commit 874a3b1f1f1da21adf8fa682aa082efb5a0efb8f
Author: caoxuewen <ca...@zte.com.cn>
Date:   2017-05-05T05:44:36Z

    Add new unit tests to ShuffleSuite

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17868: [SPARK-20607][CORE]Add new unit tests to ShuffleSuite

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/17868
  
    **[Test build #3734 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3734/testReport)** for PR 17868 at commit [`b6eff6e`](https://github.com/apache/spark/commit/b6eff6e70ed98bdc396490382c2da0c0d96acbc2).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17868: [SPARK-20607][CORE]Add new unit tests to ShuffleS...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:

    https://github.com/apache/spark/pull/17868#discussion_r114954808
  
    --- Diff: core/src/test/scala/org/apache/spark/ShuffleSuite.scala ---
    @@ -271,25 +271,49 @@ abstract class ShuffleSuite extends SparkFunSuite with Matchers with LocalSparkC
       test("[SPARK-4085] rerun map stage if reduce stage cannot find its local shuffle file") {
         val myConf = conf.clone().set("spark.test.noStageRetry", "false")
         sc = new SparkContext("local", "test", myConf)
    -    val rdd = sc.parallelize(1 to 10, 2).map((_, 1)).reduceByKey(_ + _)
    +    val rdd = sc.parallelize(1 to 10, 1).map((_, 1)).reduceByKey(_ + _)
         rdd.count()
     
         // Delete one of the local shuffle blocks.
         val hashFile = sc.env.blockManager.diskBlockManager.getFile(new ShuffleBlockId(0, 0, 0))
         val sortFile = sc.env.blockManager.diskBlockManager.getFile(new ShuffleDataBlockId(0, 0, 0))
    -    assert(hashFile.exists() || sortFile.exists())
    +    val indexFile = sc.env.blockManager.diskBlockManager.getFile(new ShuffleIndexBlockId(0, 0, 0))
    +    assert(hashFile.exists() || (sortFile.exists() && indexFile.exists()))
     
         if (hashFile.exists()) {
           hashFile.delete()
         }
         if (sortFile.exists()) {
           sortFile.delete()
         }
    +    if (indexFile.exists()) {
    +      indexFile.delete()
    +    }
     
         // This count should retry the execution of the previous stage and rerun shuffle.
         rdd.count()
       }
     
    +  test("cannot find its local shuffle file if no execution of the stage and rerun shuffle") {
    +    val myConf = conf.clone().set("spark.test.noStageRetry", "true")
    +    sc = new SparkContext("local", "test", myConf)
    +    val rdd = sc.parallelize(1 to 10, 1).map((_, 1)).reduceByKey(_ + _)
    +
    +    // Cannot find one of the local shuffle blocks.
    +    val hashFile = sc.env.blockManager.diskBlockManager.getFile(new ShuffleBlockId(0, 0, 0))
    +    val sortFile = sc.env.blockManager.diskBlockManager.getFile(new ShuffleDataBlockId(0, 0, 0))
    +    val indexFile = sc.env.blockManager.diskBlockManager.getFile(new ShuffleIndexBlockId(0, 0, 0))
    +    assert(!hashFile.exists() && !sortFile.exists() && !indexFile.exists())
    +
    +     rdd.count()
    --- End diff --
    
    Indent is off


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17868: [SPARK-20607][CORE]Add new unit tests to ShuffleSuite

Posted by heary-cao <gi...@git.apache.org>.
Github user heary-cao commented on the issue:

    https://github.com/apache/spark/pull/17868
  
    @srowen 
    I have modify this` Scala style`.
    reset please.
    thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17868: [SPARK-20607][CORE]Add new unit tests to ShuffleS...

Posted by heary-cao <gi...@git.apache.org>.
Github user heary-cao commented on a diff in the pull request:

    https://github.com/apache/spark/pull/17868#discussion_r114961998
  
    --- Diff: core/src/test/scala/org/apache/spark/ShuffleSuite.scala ---
    @@ -271,25 +271,49 @@ abstract class ShuffleSuite extends SparkFunSuite with Matchers with LocalSparkC
       test("[SPARK-4085] rerun map stage if reduce stage cannot find its local shuffle file") {
         val myConf = conf.clone().set("spark.test.noStageRetry", "false")
         sc = new SparkContext("local", "test", myConf)
    -    val rdd = sc.parallelize(1 to 10, 2).map((_, 1)).reduceByKey(_ + _)
    +    val rdd = sc.parallelize(1 to 10, 1).map((_, 1)).reduceByKey(_ + _)
    --- End diff --
    
    @srowen 
    I think it over access to shuffle the data file with the index file closer to the test target. so modify it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17868: [SPARK-20607][CORE]Add new unit tests to ShuffleSuite

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/17868
  
    **[Test build #3734 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3734/testReport)** for PR 17868 at commit [`b6eff6e`](https://github.com/apache/spark/commit/b6eff6e70ed98bdc396490382c2da0c0d96acbc2).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17868: [SPARK-20607][CORE]Add new unit tests to ShuffleSuite

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/17868
  
    **[Test build #3720 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3720/testReport)** for PR 17868 at commit [`b6eff6e`](https://github.com/apache/spark/commit/b6eff6e70ed98bdc396490382c2da0c0d96acbc2).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17868: [SPARK-20607][CORE]Add new unit tests to ShuffleSuite

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:

    https://github.com/apache/spark/pull/17868
  
    merged to master


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17868: [SPARK-20607][CORE]Add new unit tests to ShuffleS...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:

    https://github.com/apache/spark/pull/17868#discussion_r114954827
  
    --- Diff: core/src/test/scala/org/apache/spark/ShuffleSuite.scala ---
    @@ -271,25 +271,49 @@ abstract class ShuffleSuite extends SparkFunSuite with Matchers with LocalSparkC
       test("[SPARK-4085] rerun map stage if reduce stage cannot find its local shuffle file") {
         val myConf = conf.clone().set("spark.test.noStageRetry", "false")
         sc = new SparkContext("local", "test", myConf)
    -    val rdd = sc.parallelize(1 to 10, 2).map((_, 1)).reduceByKey(_ + _)
    +    val rdd = sc.parallelize(1 to 10, 1).map((_, 1)).reduceByKey(_ + _)
    --- End diff --
    
    Why this change?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17868: [SPARK-20607][CORE]Add new unit tests to ShuffleS...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/spark/pull/17868


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17868: [SPARK-20607][CORE]Add new unit tests to ShuffleS...

Posted by heary-cao <gi...@git.apache.org>.
Github user heary-cao commented on a diff in the pull request:

    https://github.com/apache/spark/pull/17868#discussion_r114965112
  
    --- Diff: core/src/test/scala/org/apache/spark/ShuffleSuite.scala ---
    @@ -271,25 +271,49 @@ abstract class ShuffleSuite extends SparkFunSuite with Matchers with LocalSparkC
       test("[SPARK-4085] rerun map stage if reduce stage cannot find its local shuffle file") {
         val myConf = conf.clone().set("spark.test.noStageRetry", "false")
         sc = new SparkContext("local", "test", myConf)
    -    val rdd = sc.parallelize(1 to 10, 2).map((_, 1)).reduceByKey(_ + _)
    +    val rdd = sc.parallelize(1 to 10, 1).map((_, 1)).reduceByKey(_ + _)
         rdd.count()
     
         // Delete one of the local shuffle blocks.
         val hashFile = sc.env.blockManager.diskBlockManager.getFile(new ShuffleBlockId(0, 0, 0))
         val sortFile = sc.env.blockManager.diskBlockManager.getFile(new ShuffleDataBlockId(0, 0, 0))
    -    assert(hashFile.exists() || sortFile.exists())
    +    val indexFile = sc.env.blockManager.diskBlockManager.getFile(new ShuffleIndexBlockId(0, 0, 0))
    +    assert(hashFile.exists() || (sortFile.exists() && indexFile.exists()))
     
         if (hashFile.exists()) {
           hashFile.delete()
         }
         if (sortFile.exists()) {
           sortFile.delete()
         }
    +    if (indexFile.exists()) {
    +      indexFile.delete()
    +    }
     
         // This count should retry the execution of the previous stage and rerun shuffle.
         rdd.count()
       }
     
    +  test("cannot find its local shuffle file if no execution of the stage and rerun shuffle") {
    +    val myConf = conf.clone().set("spark.test.noStageRetry", "true")
    +    sc = new SparkContext("local", "test", myConf)
    --- End diff --
    
    Has been modified.
    please review it again.
    thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17868: [SPARK-20607][CORE]Add new unit tests to ShuffleSuite

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/17868
  
    **[Test build #3725 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3725/testReport)** for PR 17868 at commit [`b6eff6e`](https://github.com/apache/spark/commit/b6eff6e70ed98bdc396490382c2da0c0d96acbc2).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17868: [SPARK-20607][CORE]Add new unit tests to ShuffleSuite

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/17868
  
    **[Test build #3713 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3713/testReport)** for PR 17868 at commit [`130d597`](https://github.com/apache/spark/commit/130d5970c2e378c5d0009ef602dc747ce3f7e16d).
     * This patch **fails Scala style tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17868: [SPARK-20607][CORE]Add new unit tests to ShuffleS...

Posted by heary-cao <gi...@git.apache.org>.
Github user heary-cao commented on a diff in the pull request:

    https://github.com/apache/spark/pull/17868#discussion_r114964886
  
    --- Diff: core/src/test/scala/org/apache/spark/ShuffleSuite.scala ---
    @@ -271,25 +271,49 @@ abstract class ShuffleSuite extends SparkFunSuite with Matchers with LocalSparkC
       test("[SPARK-4085] rerun map stage if reduce stage cannot find its local shuffle file") {
         val myConf = conf.clone().set("spark.test.noStageRetry", "false")
         sc = new SparkContext("local", "test", myConf)
    -    val rdd = sc.parallelize(1 to 10, 2).map((_, 1)).reduceByKey(_ + _)
    +    val rdd = sc.parallelize(1 to 10, 1).map((_, 1)).reduceByKey(_ + _)
    --- End diff --
    
    Sorry,
    When debugging use value, forget the back.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17868: [SPARK-20607][CORE]Add new unit tests to ShuffleS...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:

    https://github.com/apache/spark/pull/17868#discussion_r114954790
  
    --- Diff: core/src/test/scala/org/apache/spark/ShuffleSuite.scala ---
    @@ -271,25 +271,49 @@ abstract class ShuffleSuite extends SparkFunSuite with Matchers with LocalSparkC
       test("[SPARK-4085] rerun map stage if reduce stage cannot find its local shuffle file") {
         val myConf = conf.clone().set("spark.test.noStageRetry", "false")
         sc = new SparkContext("local", "test", myConf)
    -    val rdd = sc.parallelize(1 to 10, 2).map((_, 1)).reduceByKey(_ + _)
    +    val rdd = sc.parallelize(1 to 10, 1).map((_, 1)).reduceByKey(_ + _)
         rdd.count()
     
         // Delete one of the local shuffle blocks.
         val hashFile = sc.env.blockManager.diskBlockManager.getFile(new ShuffleBlockId(0, 0, 0))
         val sortFile = sc.env.blockManager.diskBlockManager.getFile(new ShuffleDataBlockId(0, 0, 0))
    -    assert(hashFile.exists() || sortFile.exists())
    +    val indexFile = sc.env.blockManager.diskBlockManager.getFile(new ShuffleIndexBlockId(0, 0, 0))
    +    assert(hashFile.exists() || (sortFile.exists() && indexFile.exists()))
     
         if (hashFile.exists()) {
           hashFile.delete()
         }
         if (sortFile.exists()) {
           sortFile.delete()
         }
    +    if (indexFile.exists()) {
    +      indexFile.delete()
    +    }
     
         // This count should retry the execution of the previous stage and rerun shuffle.
         rdd.count()
       }
     
    +  test("cannot find its local shuffle file if no execution of the stage and rerun shuffle") {
    +    val myConf = conf.clone().set("spark.test.noStageRetry", "true")
    +    sc = new SparkContext("local", "test", myConf)
    --- End diff --
    
    You aren't closing the existing sc here


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17868: [CORE]Add new unit tests to ShuffleSuite

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/17868
  
    Can one of the admins verify this patch?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #17868: [SPARK-20607][CORE]Add new unit tests to ShuffleS...

Posted by heary-cao <gi...@git.apache.org>.
Github user heary-cao commented on a diff in the pull request:

    https://github.com/apache/spark/pull/17868#discussion_r114964280
  
    --- Diff: core/src/test/scala/org/apache/spark/ShuffleSuite.scala ---
    @@ -271,25 +271,49 @@ abstract class ShuffleSuite extends SparkFunSuite with Matchers with LocalSparkC
       test("[SPARK-4085] rerun map stage if reduce stage cannot find its local shuffle file") {
         val myConf = conf.clone().set("spark.test.noStageRetry", "false")
         sc = new SparkContext("local", "test", myConf)
    -    val rdd = sc.parallelize(1 to 10, 2).map((_, 1)).reduceByKey(_ + _)
    +    val rdd = sc.parallelize(1 to 10, 1).map((_, 1)).reduceByKey(_ + _)
         rdd.count()
     
         // Delete one of the local shuffle blocks.
         val hashFile = sc.env.blockManager.diskBlockManager.getFile(new ShuffleBlockId(0, 0, 0))
         val sortFile = sc.env.blockManager.diskBlockManager.getFile(new ShuffleDataBlockId(0, 0, 0))
    -    assert(hashFile.exists() || sortFile.exists())
    +    val indexFile = sc.env.blockManager.diskBlockManager.getFile(new ShuffleIndexBlockId(0, 0, 0))
    +    assert(hashFile.exists() || (sortFile.exists() && indexFile.exists()))
     
         if (hashFile.exists()) {
           hashFile.delete()
         }
         if (sortFile.exists()) {
           sortFile.delete()
         }
    +    if (indexFile.exists()) {
    +      indexFile.delete()
    +    }
     
         // This count should retry the execution of the previous stage and rerun shuffle.
         rdd.count()
       }
     
    +  test("cannot find its local shuffle file if no execution of the stage and rerun shuffle") {
    +    val myConf = conf.clone().set("spark.test.noStageRetry", "true")
    +    sc = new SparkContext("local", "test", myConf)
    +    val rdd = sc.parallelize(1 to 10, 1).map((_, 1)).reduceByKey(_ + _)
    +
    +    // Cannot find one of the local shuffle blocks.
    +    val hashFile = sc.env.blockManager.diskBlockManager.getFile(new ShuffleBlockId(0, 0, 0))
    +    val sortFile = sc.env.blockManager.diskBlockManager.getFile(new ShuffleDataBlockId(0, 0, 0))
    +    val indexFile = sc.env.blockManager.diskBlockManager.getFile(new ShuffleIndexBlockId(0, 0, 0))
    +    assert(!hashFile.exists() && !sortFile.exists() && !indexFile.exists())
    +
    +     rdd.count()
    --- End diff --
    
    sorry,
    I have modify it.
    thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #17868: [SPARK-20607][CORE]Add new unit tests to ShuffleSuite

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/17868
  
    **[Test build #3713 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3713/testReport)** for PR 17868 at commit [`130d597`](https://github.com/apache/spark/commit/130d5970c2e378c5d0009ef602dc747ce3f7e16d).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org