[jira] [Commented] (BEAM-22) DirectPipelineRunner: support for unbounded collections

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-22?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176604#comment-15176604 ] 

ASF GitHub Bot commented on BEAM-22:
------------------------------------

GitHub user tgroh opened a pull request:

    https://github.com/apache/incubator-beam/pull/11

    [BEAM-22] Implement InProcessEvaluationContext

    This is the primary "global state" object for the evaluation of a
    Pipeline using the InProcessPipelineRunner, and is responsible for
    properly routing information about the state of the pipeline to
    transform evaluators.
    
    Remove the InProcessEvaluationContext from the InProcessPipelineRunner
    class, and implement as a class directly. Fix associated imports.
    
    Split from the first commit in #3 

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/tgroh/incubator-beam ippr_evaluation_context

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-beam/pull/11.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #11
    
----
commit 497975272a8028cfcb2db9ffa973e86dff3f36d5
Author: Thomas Groh <tg...@google.com>
Date:   2016-02-27T01:28:37Z

    Implement InProcessEvaluationContext
    
    This is the primary "global state" object for the evaluation of a
    Pipeline using the InProcessPipelineRunner, and is responsible for
    properly routing information about the state of the pipeline to
    transform evaluators.
    
    Remove the InProcessEvaluationContext from the InProcessPipelineRunner
    class, and implement as a class directly. Fix associated imports.

----


> DirectPipelineRunner: support for unbounded collections
> -------------------------------------------------------
>
>                 Key: BEAM-22
>                 URL: https://issues.apache.org/jira/browse/BEAM-22
>             Project: Beam
>          Issue Type: Improvement
>          Components: runner-direct
>            Reporter: Davor Bonaci
>            Assignee: Thomas Groh
>
> DirectPipelineRunner currently runs over bounded PCollections only, and implements only a portion of the Beam Model.
> We should improve it to faithfully implement the full Beam Model, such as add ability to run over unbounded PCollections, and better resemble execution model in a distributed system.
> This further enables features such as a testing source which may simulate late data and test triggers in the pipeline. Finally, we may want to expose an option to select between "debug" (single threaded), "chaos monkey" (test as many model requirements as possible), and "performance" (multi-threaded).
> more testing (chaos monkey) 
> Once this is done, we should update this StackOverflow question:
> http://stackoverflow.com/questions/35350113/testing-triggers-with-processing-time/35401426#35401426



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)