You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@spark.apache.org by do...@apache.org on 2022/03/01 19:34:03 UTC

[spark] branch master updated (ccb8af6 -> 42f118a)

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git.


    from ccb8af6  [SPARK-38188][K8S] Support `spark.kubernetes.job.queue`
     add 42f118a  [SPARK-33206][CORE] Fix shuffle index cache weight calculation for small index files

No new revisions were added by this update.

Summary of changes:
 .../spark/network/shuffle/ExecutorDiskUtils.java   |  7 +--
 .../shuffle/ExternalShuffleBlockResolver.java      | 46 ++++++++------
 .../network/shuffle/RemoteBlockPushResolver.java   | 48 ++++++++-------
 .../network/shuffle/ShuffleIndexInformation.java   | 23 ++++---
 .../shuffle/RemoteBlockPushResolverSuite.java      |  4 +-
 .../shuffle/ShuffleIndexInformationSuite.java      | 71 ++++++++++++++++++++++
 .../network/shuffle/TestShuffleDataContext.java    | 16 ++---
 .../spark/shuffle/IndexShuffleBlockResolver.scala  |  9 ++-
 .../org/apache/spark/storage/BlockManager.scala    |  2 +-
 .../apache/spark/storage/DiskBlockManager.scala    |  4 +-
 .../apache/spark/storage/BlockManagerSuite.scala   |  3 +-
 11 files changed, 161 insertions(+), 72 deletions(-)
 create mode 100644 common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/ShuffleIndexInformationSuite.java

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org