You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by jkff <gi...@git.apache.org> on 2016/11/12 01:08:48 UTC

[GitHub] incubator-beam pull request #1353: Liberates ReduceFnRunner from WindowingIn...

GitHub user jkff opened a pull request:

    https://github.com/apache/incubator-beam/pull/1353

    Liberates ReduceFnRunner from WindowingInternals, and lets WindowingInternals do windowed side outputs

    - Introduces WindowingInternals.sideOutputWindowedValue (will be necessary for Splittable DoFn)
    - Implements it properly in all runners (required some minor refactoring in Spark and Flink ProcessContext implementations)
    - Introduces "OutputWindowedValue" interface and "SideInputAccess" interfaces, and uses them in ReduceFnRunner instead of directly using WindowingInternals.
    - Introduces adapters from WindowingInternals to these two interfaces, for use in OldDoFn contexts
    - Moves some StateContext functions into ReduceFnContextFactory, because they make more sense in runners-core than in sdk (because they are only invoked by different runners). The only remaining StateContexts function is nullContext, but I couldn't figure out an easy way to move it into runners-core and gave up (however in fact I'm not sure its current usages are correct at all...)
    
    R: @kennknowles (for bulk of the code and as committer)
    CC: @aljoscha @amitsela (for the minor refactorings in respective runners)

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/jkff/incubator-beam reducefn-windowing-internals

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-beam/pull/1353.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1353
    
----
commit 14dd49e02ab2eabb7c7e42cb1e9cab29f7cea44e
Author: Eugene Kirpichov <ki...@google.com>
Date:   2016-11-11T02:40:53Z

    Liberates ReduceFnRunner from WindowingInternals

commit d6c62454bb822063c863d2bcd1f2436453b94864
Author: Eugene Kirpichov <ki...@google.com>
Date:   2016-11-12T00:37:42Z

    Refactor FlinkProcessContext more cleanly into single- and multi-output versions

commit 14e5847f738b3d89808d5d612684fcddffd51b7f
Author: Eugene Kirpichov <ki...@google.com>
Date:   2016-11-12T00:57:27Z

    Refactor SparkProcessContext more cleanly into single- and multi-output versions

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-beam pull request #1353: Liberates ReduceFnRunner from WindowingIn...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/incubator-beam/pull/1353


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---