You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tez.apache.org by ab...@apache.org on 2022/03/16 18:18:46 UTC
[tez] branch master updated (132ea4c -> 20873a3)
This is an automated email from the ASF dual-hosted git repository.
abstractdog pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/tez.git.
from 132ea4c TEZ-4359: ShuffleHandler: Make sure of properly releasing netty reference counted objects (#194) (Mark Bathori reviewed by Laszlo Bodor)
add 20873a3 TEZ-3363: Delete intermediate data at the vertex level for Shuffle Handler (#60) (Syed Shameerur Rahman reviewed by Laszlo Bodor)
No new revisions were added by this update.
Summary of changes:
.../org/apache/tez/dag/api/TezConfiguration.java | 20 +-
.../apache/tez/common/DagContainerLauncher.java | 6 +
.../java/org/apache/tez/dag/app/DAGAppMaster.java | 4 +
.../tez/dag/app/dag/event/VertexEventType.java | 1 +
...tUpdate.java => VertexShuffleDataDeletion.java} | 28 +-
.../org/apache/tez/dag/app/dag/impl/DAGImpl.java | 8 +
.../apache/tez/dag/app/dag/impl/VertexImpl.java | 72 ++-
.../dag/impl/VertexShuffleDataDeletionContext.java | 96 ++++
.../dag/app/launcher/ContainerLauncherManager.java | 8 +
.../dag/app/launcher/ContainerLauncherWrapper.java | 9 +
.../tez/dag/app/launcher/DeletionTracker.java | 7 +
.../tez/dag/app/launcher/DeletionTrackerImpl.java | 23 +
.../dag/app/launcher/LocalContainerLauncher.java | 15 +-
.../dag/app/launcher/TezContainerLauncherImpl.java | 16 +-
...leteRunnable.java => VertexDeleteRunnable.java} | 43 +-
.../tez/dag/app/dag/impl/TestVertexImpl.java | 529 ++++++++++++++++++---
.../app/launcher/TestContainerLauncherWrapper.java | 2 +-
.../org/apache/tez/auxservices/ShuffleHandler.java | 48 ++
.../apache/tez/auxservices/TestShuffleHandler.java | 78 +++
.../runtime/library/common/TezRuntimeUtils.java | 19 +
20 files changed, 935 insertions(+), 97 deletions(-)
copy tez-dag/src/main/java/org/apache/tez/dag/app/dag/event/{TaskAttemptEventTezEventUpdate.java => VertexShuffleDataDeletion.java} (60%)
create mode 100644 tez-dag/src/main/java/org/apache/tez/dag/app/dag/impl/VertexShuffleDataDeletionContext.java
copy tez-dag/src/main/java/org/apache/tez/dag/app/launcher/{DagDeleteRunnable.java => VertexDeleteRunnable.java} (60%)