You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/08/15 14:37:20 UTC
[jira] [Commented] (BEAM-551) Support Dynamic PipelineOptions

    [ https://issues.apache.org/jira/browse/BEAM-551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421067#comment-15421067 ] 

ASF GitHub Bot commented on BEAM-551:
-------------------------------------

GitHub user sammcveety opened a pull request:

    https://github.com/apache/incubator-beam/pull/827

    [BEAM-551] Add ValueProvider class

    Hi @lukecwik , can you please take a look?
    
    Currently implemented for ValueProvider<String> only.  Tests demonstrate intended functionality, once the questions below are answered:
    
    - What's the best way to avoid threading issues with the static options in test?
    - Is it reasonable to continue down the road of special-casing getter/setter validation for this class?
    - What would be the preferred way to annotate T for ValueProvider<T> to work around type erasure?

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/sammcveety/incubator-beam sgmc/valueprovider

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-beam/pull/827.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #827
    
----
commit 867a080b98f585db2687d9aaf7d488882de8c813
Author: sammcveety <sa...@gmail.com>
Date:   2016-08-14T00:45:58Z

    Merge pull request #1 from apache/master
    
    Update my fork.

commit bb7aea8db07b3a7ae988f75888716dd865e07e08
Author: sammcveety <sa...@gmail.com>
Date:   2016-08-14T00:48:18Z

    Create ValueProvider.java

commit 5330800d9d5b54b897c53cb8ea40c6331fe3c36a
Author: sammcveety <sa...@gmail.com>
Date:   2016-08-14T00:48:54Z

    Update ValueProvider.java

commit 9b9b17c2338612b4030d1c4a36f8fb4c5c179c2e
Author: sammcveety <sa...@gmail.com>
Date:   2016-08-14T00:50:56Z

    Update ProxyInvocationHandler.java

commit b54832bd0c28691d1e6087b4901fb6ae11ebd2c9
Author: sammcveety <sa...@gmail.com>
Date:   2016-08-14T00:54:33Z

    Create ValueProviderTest.java

commit e2d16a7ad66676a099d4160ecd4c31c369acd46e
Author: sammcveety <sa...@gmail.com>
Date:   2016-08-14T00:56:41Z

    Update ValueProviderTest.java

commit afbe8157f711fca00ed37afbe461edd4ac78e30f
Author: sammcveety <sa...@gmail.com>
Date:   2016-08-14T16:34:23Z

    Update ProxyInvocationHandler.java

commit e111045ae1125ed49ea1f26cdde31b3c1583ed2d
Author: sammcveety <sa...@gmail.com>
Date:   2016-08-14T16:58:29Z

    Update ValueProviderTest.java

commit 0bff463a71c4f4b3ad13edd924fd03226a915a61
Author: sammcveety <sa...@gmail.com>
Date:   2016-08-14T20:11:49Z

    Add tests.

commit c4f3acc427b828f0f0254f83e84690b14c0a0f98
Author: sammcveety <sa...@gmail.com>
Date:   2016-08-14T21:42:56Z

    Iterations on tests.

commit f19435b688fed41d873b85ac7750d923025256a9
Author: sammcveety <sa...@gmail.com>
Date:   2016-08-15T01:32:37Z

    Proposal for how to handle getters/setters.

----


> Support Dynamic PipelineOptions
> -------------------------------
>
>                 Key: BEAM-551
>                 URL: https://issues.apache.org/jira/browse/BEAM-551
>             Project: Beam
>          Issue Type: New Feature
>          Components: beam-model
>            Reporter: Sam McVeety
>            Assignee: Frances Perry
>            Priority: Minor
>
> During the graph construction phase, the given SDK generates an initial
> execution graph for the program.  At execution time, this graph is
> executed, either locally or by a service.  Currently, Beam only supports
> parameterization at graph construction time.  Both Flink and Spark supply
> functionality that allows a pre-compiled job to be run without SDK
> interaction with updated runtime parameters.
> In its current incarnation, Dataflow can read values of PipelineOptions at
> job submission time, but this requires the presence of an SDK to properly
> encode these values into the job.  We would like to build a common layer
> into the Beam model so that these dynamic options can be properly provided
> to jobs.
> Please see
> https://docs.google.com/document/d/1I-iIgWDYasb7ZmXbGBHdok_IK1r1YAJ90JG5Fz0_28o/edit
> for the high-level model, and
> https://docs.google.com/document/d/17I7HeNQmiIfOJi0aI70tgGMMkOSgGi8ZUH-MOnFatZ8/edit
> for
> the specific API proposal.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)