You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@celeborn.apache.org by zh...@apache.org on 2023/04/17 06:44:29 UTC

[incubator-celeborn] branch main updated (938aec0e9 -> 412d10b7d)

This is an automated email from the ASF dual-hosted git repository.

zhouky pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-celeborn.git


    from 938aec0e9 [CELEBORN-528][REFACTOR] limitZeroInFlight should show inflight target (#1433)
     add 412d10b7d [CELEBORN-479][FLINK] support stopTrackingAndReleasePartitions when worker is not available (#1405)

No new revisions were added by this update.

Summary of changes:
 .../plugin/flink/RemoteShuffleDescriptor.java      |   9 +
 .../celeborn/plugin/flink/RemoteShuffleMaster.java |  24 +++
 .../plugin/flink/ShuffleResourceTracker.java       | 210 +++++++++++++++++++++
 .../flink/RemoteShuffleResultPartitionSuiteJ.java  |   1 +
 .../plugin/flink/ShuffleResourceTrackerTest.java   | 127 +++++++++++++
 .../client/listener/WorkerStatusListener.java      |  19 +-
 .../celeborn/client/listener/WorkersStatus.java    |  16 +-
 .../celeborn/client/ApplicationHeartbeater.scala   |   2 +-
 .../org/apache/celeborn/client/CommitManager.scala |  44 +++--
 .../apache/celeborn/client/LifecycleManager.scala  |  16 +-
 .../celeborn/client/WorkerStatusTracker.scala      |  41 +++-
 common/src/main/proto/TransportMessages.proto      |   2 +-
 .../org/apache/celeborn/common/CelebornConf.scala  |  12 ++
 .../common/meta/ShufflePartitionLocationInfo.scala |   6 +
 .../common/protocol/message/ControlMessages.scala  |  16 +-
 .../apache/celeborn/common/util/PbSerDeUtils.scala |   2 +-
 .../celeborn/service/deploy/master/Master.scala    |   7 +-
 17 files changed, 502 insertions(+), 52 deletions(-)
 create mode 100644 client-flink/flink-1.14/src/main/java/org/apache/celeborn/plugin/flink/ShuffleResourceTracker.java
 create mode 100644 client-flink/flink-1.14/src/test/java/org/apache/celeborn/plugin/flink/ShuffleResourceTrackerTest.java
 copy common/src/main/java/org/apache/celeborn/common/network/util/IOMode.java => client/src/main/java/org/apache/celeborn/client/listener/WorkerStatusListener.java (68%)
 copy common/src/main/java/org/apache/celeborn/common/network/server/memory/ReadBufferListener.java => client/src/main/java/org/apache/celeborn/client/listener/WorkersStatus.java (67%)