[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

[yucai <gi...@git.apache.org>]

Github user yucai commented on a diff in the pull request:

    https://github.com/apache/spark/pull/19788#discussion_r165408800
  
    --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockHandler.java ---
    @@ -203,22 +203,23 @@ private ShuffleMetrics() {
           this.appId = appId;
           this.execId = execId;
           String[] blockId0Parts = blockIds[0].split("_");
    -      if (blockId0Parts.length != 4 || !blockId0Parts[0].equals("shuffle")) {
    +      if (blockId0Parts.length != 5 || !blockId0Parts[0].equals("shuffle")) {
             throw new IllegalArgumentException("Unexpected shuffle block id format: " + blockIds[0]);
           }
           this.shuffleId = Integer.parseInt(blockId0Parts[1]);
    -      mapIdAndReduceIds = new int[2 * blockIds.length];
    +      mapIdAndReduceIds = new int[3 * blockIds.length];
    --- End diff --
    
    Thanks, fixed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org