You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nemo.apache.org by jo...@apache.org on 2018/11/01 02:03:15 UTC
[incubator-nemo] branch maven-artifact updated (49266b5 -> 85d2507)
This is an automated email from the ASF dual-hosted git repository.
jooykim pushed a change to branch maven-artifact
in repository https://gitbox.apache.org/repos/asf/incubator-nemo.git.
from 49266b5 Merge branch 'master' into maven-artifact
add 04d6eb4 [NEMO-230] Emit collected data when receiving watermark in GroupByKeyAndWindowTransform (#135)
add 53522c6 [NEMO-237] Refactor ParentTaskDataFetcher to emit streaming data and watermark (#138)
add 7858051 [NEMO-232] Implement InputWatermarkManager (#137)
new 85d2507 Merge branch 'master' into maven-artifact
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../GroupByKeyAndWindowDoFnTransform.java | 165 ++++++++++---
.../frontend/beam/transform/DoFnTransformTest.java | 35 ---
.../GroupByKeyAndWindowDoFnTransformTest.java | 176 +++++++++++++
.../beam/transform/TestOutputCollector.java | 69 ++++++
pom.xml | 16 +-
.../runtime/executor/data/PipeManagerWorker.java | 8 +-
.../datatransfer/DataFetcherOutputCollector.java | 1 +
.../datatransfer/InputWatermarkManager.java | 46 ++++
.../datatransfer/MultiInputWatermarkManager.java | 90 +++++++
.../datatransfer/NextIntraTaskOperatorInfo.java | 59 +++++
.../OperatorVertexOutputCollector.java | 35 ++-
...eader.java => SingleInputWatermarkManager.java} | 35 +--
.../task/MultiThreadParentTaskDataFetcher.java | 150 +++++++++++
.../executor/task/ParentTaskDataFetcher.java | 1 -
.../nemo/runtime/executor/task/TaskExecutor.java | 111 +++++++--
.../datatransfer/InputWatermarkManagerTest.java | 88 +++++++
.../runtime/executor/task/TaskExecutorTest.java | 274 +++++++++++++++------
17 files changed, 1145 insertions(+), 214 deletions(-)
create mode 100644 compiler/test/src/test/java/org/apache/nemo/compiler/frontend/beam/transform/GroupByKeyAndWindowDoFnTransformTest.java
create mode 100644 compiler/test/src/test/java/org/apache/nemo/compiler/frontend/beam/transform/TestOutputCollector.java
create mode 100644 runtime/executor/src/main/java/org/apache/nemo/runtime/executor/datatransfer/InputWatermarkManager.java
create mode 100644 runtime/executor/src/main/java/org/apache/nemo/runtime/executor/datatransfer/MultiInputWatermarkManager.java
create mode 100644 runtime/executor/src/main/java/org/apache/nemo/runtime/executor/datatransfer/NextIntraTaskOperatorInfo.java
copy runtime/executor/src/main/java/org/apache/nemo/runtime/executor/datatransfer/{InputReader.java => SingleInputWatermarkManager.java} (53%)
create mode 100644 runtime/executor/src/main/java/org/apache/nemo/runtime/executor/task/MultiThreadParentTaskDataFetcher.java
create mode 100644 runtime/executor/src/test/java/org/apache/nemo/runtime/executor/datatransfer/InputWatermarkManagerTest.java
[incubator-nemo] 01/01: Merge branch 'master' into maven-artifact
Posted by jo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
jooykim pushed a commit to branch maven-artifact
in repository https://gitbox.apache.org/repos/asf/incubator-nemo.git
commit 85d25073a371dbcecbc140d6a218254a666a881e
Merge: 49266b5 7858051
Author: Joo Yeon Kim <jo...@apache.org>
AuthorDate: Thu Nov 1 11:02:27 2018 +0900
Merge branch 'master' into maven-artifact
.../GroupByKeyAndWindowDoFnTransform.java | 165 ++++++++++---
.../frontend/beam/transform/DoFnTransformTest.java | 35 ---
.../GroupByKeyAndWindowDoFnTransformTest.java | 176 +++++++++++++
.../beam/transform/TestOutputCollector.java | 69 ++++++
pom.xml | 16 +-
.../runtime/executor/data/PipeManagerWorker.java | 8 +-
.../datatransfer/DataFetcherOutputCollector.java | 1 +
.../datatransfer/InputWatermarkManager.java | 46 ++++
.../datatransfer/MultiInputWatermarkManager.java | 90 +++++++
.../datatransfer/NextIntraTaskOperatorInfo.java | 59 +++++
.../OperatorVertexOutputCollector.java | 35 ++-
.../datatransfer/SingleInputWatermarkManager.java | 46 ++++
.../task/MultiThreadParentTaskDataFetcher.java | 150 +++++++++++
.../executor/task/ParentTaskDataFetcher.java | 1 -
.../nemo/runtime/executor/task/TaskExecutor.java | 111 +++++++--
.../datatransfer/InputWatermarkManagerTest.java | 88 +++++++
.../runtime/executor/task/TaskExecutorTest.java | 274 +++++++++++++++------
17 files changed, 1173 insertions(+), 197 deletions(-)