You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@spark.apache.org by va...@apache.org on 2019/09/11 00:30:50 UTC

[spark] branch master updated (fa75db2 -> 7f36cd2)

This is an automated email from the ASF dual-hosted git repository.

vanzin pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.


    from fa75db2  [SPARK-29026][SQL] Improve error message in `schemaFor` in trait without companion object constructor
     add 7f36cd2  [SPARK-28570][CORE][SHUFFLE] Make UnsafeShuffleWriter use the new API

No new revisions were added by this update.

Summary of changes:
 .../shuffle/api/ShuffleExecutorComponents.java     |  31 +-
 .../api/SingleSpillShuffleMapOutputWriter.java     |  12 +-
 .../spark/shuffle/sort/UnsafeShuffleWriter.java    | 352 +++++++++++----------
 .../io/LocalDiskShuffleExecutorComponents.java     |  15 +
 .../sort/io/LocalDiskShuffleMapOutputWriter.java   |  24 +-
 .../io/LocalDiskSingleSpillMapOutputWriter.java    |  55 ++++
 .../org/apache/spark/internal/config/package.scala |   2 +-
 .../spark/shuffle/sort/SortShuffleManager.scala    |   4 +-
 .../shuffle/sort/UnsafeShuffleWriterSuite.java     |  54 ++--
 9 files changed, 347 insertions(+), 202 deletions(-)
 copy common/kvstore/src/main/java/org/apache/spark/util/kvstore/UnsupportedStoreVersionException.java => core/src/main/java/org/apache/spark/shuffle/api/SingleSpillShuffleMapOutputWriter.java (67%)
 create mode 100644 core/src/main/java/org/apache/spark/shuffle/sort/io/LocalDiskSingleSpillMapOutputWriter.java


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org