You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Kenneth Knowles <kl...@google.com> on 2018/01/26 04:37:31 UTC
Samza Runner
Hi all,
In case you haven't noticed or followed, there's a new runner in PR: Samza!
https://github.com/apache/beam/pull/4340
It has been under review and revision for some time. In local mode it
passes a solid suite of ValidatesRunner tests (I don't have a Samza
deployment handy to test non-local).
Given all this, I am ready to put it on a feature branch where it can
mature further, and we can build out our CI for it, etc, until we agree it
is ready for master.
Kenn
Re: Samza Runner
Posted by Henning Rohde <he...@google.com>.
+1 Exciting to see a new runner!
On Thu, Jan 25, 2018 at 8:56 PM, Jesse Anderson <je...@bigdatainstitute.io>
wrote:
> Excellent!
>
> On Fri, Jan 26, 2018, 5:37 AM Kenneth Knowles <kl...@google.com> wrote:
>
>> Hi all,
>>
>> In case you haven't noticed or followed, there's a new runner in PR:
>> Samza!
>>
>> https://github.com/apache/beam/pull/4340
>>
>> It has been under review and revision for some time. In local mode it
>> passes a solid suite of ValidatesRunner tests (I don't have a Samza
>> deployment handy to test non-local).
>>
>> Given all this, I am ready to put it on a feature branch where it can
>> mature further, and we can build out our CI for it, etc, until we agree it
>> is ready for master.
>>
>> Kenn
>>
>
Re: Samza Runner
Posted by Jesse Anderson <je...@bigdatainstitute.io>.
Excellent!
On Fri, Jan 26, 2018, 5:37 AM Kenneth Knowles <kl...@google.com> wrote:
> Hi all,
>
> In case you haven't noticed or followed, there's a new runner in PR: Samza!
>
> https://github.com/apache/beam/pull/4340
>
> It has been under review and revision for some time. In local mode it
> passes a solid suite of ValidatesRunner tests (I don't have a Samza
> deployment handy to test non-local).
>
> Given all this, I am ready to put it on a feature branch where it can
> mature further, and we can build out our CI for it, etc, until we agree it
> is ready for master.
>
> Kenn
>
Re: ***UNCHECKED*** Re: Samza Runner
Posted by xinyu liu <xi...@gmail.com>.
Thanks Kenneth to merge the Samza BEAM runner to the feature branch! We
will work on the other items (docs, example, capability matrix ..) to get
it to the master.
Thanks,
Xinyu
On Fri, Jan 26, 2018 at 9:28 AM, Kenneth Knowles <kl...@google.com> wrote:
> Regarding merging directly to master, I agree that the code itself is
> probably just as OK as our other runners were when they joined master. So
> we should watch the rest of the bits from https://beam.apache.org/
> contribute/feature-branches/ here is how I think things are:
>
> [ ] Have at least 2 contributors interested in maintaining it, and 1
> committer interested in supporting it
> [ ] Provide both end-user and developer-facing documentation
> [x] Have at least a basic level of unit test coverage
> [x] Run all existing applicable integration tests with other Beam
> components and create additional tests as appropriate
> [~] Be able to handle a subset of the model that addresses a significant
> set of use cases, such as ‘traditional batch’ or ‘processing time
> streaming’.
> [ ] Update the capability matrix with the current status
> [ ] Add a webpage under documentation/runners
>
> So I would like to merge to the feature branch while we get docs together,
> identify interested contributors, and make sure of the integration with the
> build/test system, and maybe run through manual verification of the
> streaming game example at medium scale (anything bigger than local
> one-node).
>
> I also want to get it on a feature branch for this reason: we could
> continue to perfect it on the PR, but it is easier to have separate reviews
> for different issues identified on the big PR.
>
> Kenn
>
> On Thu, Jan 25, 2018 at 11:32 PM, Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
>> That's awesome ! Happy to see new runner.
>>
>> As the build is OK and the runner contains validation, why not simply
>> merge the
>> PR on master once 2.3.0 release branch is there ?
>>
>> It would give better visibility to this new feature and maybe attract
>> contribution in early stage.
>>
>> Regards
>> JB
>>
>> On 01/26/2018 05:37 AM, Kenneth Knowles wrote:
>> > Hi all,
>> >
>> > In case you haven't noticed or followed, there's a new runner in PR:
>> Samza!
>> >
>> > https://github.com/apache/beam/pull/4340
>> >
>> > It has been under review and revision for some time. In local mode it
>> passes a
>> > solid suite of ValidatesRunner tests (I don't have a Samza deployment
>> handy to
>> > test non-local).
>> >
>> > Given all this, I am ready to put it on a feature branch where it can
>> mature
>> > further, and we can build out our CI for it, etc, until we agree it is
>> ready for
>> > master.
>> >
>> > Kenn
>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>
>
***UNCHECKED*** Re: Samza Runner
Posted by Kenneth Knowles <kl...@google.com>.
Regarding merging directly to master, I agree that the code itself is
probably just as OK as our other runners were when they joined master. So
we should watch the rest of the bits from
https://beam.apache.org/contribute/feature-branches/ here is how I think
things are:
[ ] Have at least 2 contributors interested in maintaining it, and 1
committer interested in supporting it
[ ] Provide both end-user and developer-facing documentation
[x] Have at least a basic level of unit test coverage
[x] Run all existing applicable integration tests with other Beam
components and create additional tests as appropriate
[~] Be able to handle a subset of the model that addresses a significant
set of use cases, such as ‘traditional batch’ or ‘processing time
streaming’.
[ ] Update the capability matrix with the current status
[ ] Add a webpage under documentation/runners
So I would like to merge to the feature branch while we get docs together,
identify interested contributors, and make sure of the integration with the
build/test system, and maybe run through manual verification of the
streaming game example at medium scale (anything bigger than local
one-node).
I also want to get it on a feature branch for this reason: we could
continue to perfect it on the PR, but it is easier to have separate reviews
for different issues identified on the big PR.
Kenn
On Thu, Jan 25, 2018 at 11:32 PM, Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> That's awesome ! Happy to see new runner.
>
> As the build is OK and the runner contains validation, why not simply
> merge the
> PR on master once 2.3.0 release branch is there ?
>
> It would give better visibility to this new feature and maybe attract
> contribution in early stage.
>
> Regards
> JB
>
> On 01/26/2018 05:37 AM, Kenneth Knowles wrote:
> > Hi all,
> >
> > In case you haven't noticed or followed, there's a new runner in PR:
> Samza!
> >
> > https://github.com/apache/beam/pull/4340
> >
> > It has been under review and revision for some time. In local mode it
> passes a
> > solid suite of ValidatesRunner tests (I don't have a Samza deployment
> handy to
> > test non-local).
> >
> > Given all this, I am ready to put it on a feature branch where it can
> mature
> > further, and we can build out our CI for it, etc, until we agree it is
> ready for
> > master.
> >
> > Kenn
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: Samza Runner
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
That's awesome ! Happy to see new runner.
As the build is OK and the runner contains validation, why not simply merge the
PR on master once 2.3.0 release branch is there ?
It would give better visibility to this new feature and maybe attract
contribution in early stage.
Regards
JB
On 01/26/2018 05:37 AM, Kenneth Knowles wrote:
> Hi all,
>
> In case you haven't noticed or followed, there's a new runner in PR: Samza!
>
> https://github.com/apache/beam/pull/4340
>
> It has been under review and revision for some time. In local mode it passes a
> solid suite of ValidatesRunner tests (I don't have a Samza deployment handy to
> test non-local).
>
> Given all this, I am ready to put it on a feature branch where it can mature
> further, and we can build out our CI for it, etc, until we agree it is ready for
> master.
>
> Kenn
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com