You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by mr...@apache.org on 2022/09/09 02:06:07 UTC
[spark] branch master updated (6577c43852d -> e83aedd0f07)
This is an automated email from the ASF dual-hosted git repository.
mridulm80 pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
from 6577c43852d [SPARK-40332][PS] Implement `GroupBy.quantile`
add e83aedd0f07 [SPARK-38888][BUILD][CORE][YARN][DOCS] Add `RocksDB` support for shuffle service state store
No new revisions were added by this update.
Summary of changes:
common/network-common/pom.xml | 4 +
.../apache/spark/network/shuffledb/DBBackend.java | 4 +-
.../shuffledb/{LevelDB.java => RocksDB.java} | 36 +++--
.../{LevelDBIterator.java => RocksDBIterator.java} | 51 ++++---
.../org/apache/spark/network/util/DBProvider.java | 13 +-
.../apache/spark/network/util/RocksDBProvider.java | 170 +++++++++++++++++++++
.../org/apache/spark/internal/config/package.scala | 2 +-
.../deploy/ExternalShuffleServiceDbSuite.scala | 17 ++-
.../apache/spark/deploy/worker/WorkerSuite.scala | 19 ++-
docs/running-on-yarn.md | 10 ++
docs/spark-standalone.md | 10 ++
.../yarn/YarnShuffleAlternateNameConfigSuite.scala | 6 +
.../deploy/yarn/YarnShuffleIntegrationSuite.scala | 11 ++
.../network/yarn/YarnShuffleServiceSuite.scala | 6 +-
14 files changed, 314 insertions(+), 45 deletions(-)
copy common/network-common/src/main/java/org/apache/spark/network/shuffledb/{LevelDB.java => RocksDB.java} (63%)
copy common/network-common/src/main/java/org/apache/spark/network/shuffledb/{LevelDBIterator.java => RocksDBIterator.java} (70%)
create mode 100644 common/network-common/src/main/java/org/apache/spark/network/util/RocksDBProvider.java
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org