You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by ke...@apache.org on 2017/01/06 20:08:00 UTC
[5/5] beam git commit: This closes #1678: Refactor Dataflow
translator to decouple steps
This closes #1678: Refactor Dataflow translator to decouple steps
Reduce visibility of many Dataflow runner internals
Move some pieces of Dataflow translator to top level
Add explicit translation builder for a Step to in Dataflow translator
Project: http://git-wip-us.apache.org/repos/asf/beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/a6caa82a
Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/a6caa82a
Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/a6caa82a
Branch: refs/heads/master
Commit: a6caa82a621a0a9c1cd2cd92c26304e9f79546bb
Parents: b4d8702 33907f8
Author: Kenneth Knowles <kl...@google.com>
Authored: Fri Jan 6 11:37:07 2017 -0800
Committer: Kenneth Knowles <kl...@google.com>
Committed: Fri Jan 6 11:37:07 2017 -0800
----------------------------------------------------------------------
.../beam/runners/dataflow/AssignWindows.java | 89 +++
.../dataflow/DataflowAggregatorTransforms.java | 79 +++
.../dataflow/DataflowMetricUpdateExtractor.java | 109 ++++
.../runners/dataflow/DataflowPipelineJob.java | 2 -
.../dataflow/DataflowPipelineTranslator.java | 424 ++++++--------
.../beam/runners/dataflow/DataflowRunner.java | 68 +--
.../DataflowUnboundedReadFromBoundedSource.java | 547 +++++++++++++++++++
.../beam/runners/dataflow/ReadTranslator.java | 102 ++++
.../runners/dataflow/TransformTranslator.java | 123 +++++
.../dataflow/internal/AssignWindows.java | 89 ---
.../internal/DataflowAggregatorTransforms.java | 79 ---
.../internal/DataflowMetricUpdateExtractor.java | 109 ----
.../DataflowUnboundedReadFromBoundedSource.java | 547 -------------------
.../dataflow/internal/ReadTranslator.java | 104 ----
.../dataflow/DataflowPipelineJobTest.java | 1 -
.../DataflowPipelineTranslatorTest.java | 3 +-
.../runners/dataflow/DataflowRunnerTest.java | 8 +-
...aflowUnboundedReadFromBoundedSourceTest.java | 79 +++
...aflowUnboundedReadFromBoundedSourceTest.java | 79 ---
19 files changed, 1316 insertions(+), 1325 deletions(-)
----------------------------------------------------------------------