You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Tyler Akidau <ta...@google.com.INVALID> on 2016/03/09 23:14:29 UTC
Capability Matrix
I just filed BEAM-104 <https://issues.apache.org/jira/browse/BEAM-104>
regarding publishing a capability matrix on the Beam website. We've seeded
the spreadsheet linked there (
https://docs.google.com/spreadsheets/d/1OM077lZBARrtUi6g0X0O0PHaIbFKCD6v0djRefQRE1I/edit
)
with an initial proposed set of capabilities, as well as descriptions for
the model and Cloud Dataflow. If folks for other runners (currently Flink
and Spark) could please make sure their columns are filled out as well,
it'd be much appreciated. Also let us know if there are capabilities you
think we've missed.
Our hope is to get this up and published soon, since we've been getting a
lot of questions regarding runner capabilities, portability, etc.
-Tyler
Re: Capability Matrix
Posted by Tyler Akidau <ta...@google.com.INVALID>.
Thanks, all!
On Fri, Mar 18, 2016 at 6:46 AM Amit Sela <am...@gmail.com> wrote:
> Looks great!
> I think it's the best way to give a clear picture of capabilities for users
> and runner developers.
> And as always, Love the colours ;)
>
>
> On Fri, Mar 18, 2016 at 3:33 PM Kostas Kloudas <
> k.kloudas@data-artisans.com>
> wrote:
>
> > Great to have an overview of the available
> > runners and a comprehensible visualization of
> > the features each one supports!
> >
> > Kostas
> >
> > > On Mar 18, 2016, at 11:32 AM, Maximilian Michels <mx...@apache.org>
> wrote:
> > >
> > > Well done. The matrix provides a good basis for improving the existing
> > > runners. Moreover, new backends can use it to evaluate capabilities
> > > for creating a runner.
> > >
> > > On Fri, Mar 18, 2016 at 1:15 AM, Jean-Baptiste Onofré <jb@nanthrax.net
> >
> > wrote:
> > >> Catcha, thanks !
> > >>
> > >> Regards
> > >> JB
> > >>
> > >>
> > >> On 03/18/2016 12:51 AM, Frances Perry wrote:
> > >>>
> > >>> That's "partially". Check out the full matrix for complete details:
> > >>> http://beam.incubator.apache.org/capability-matrix/
> > >>>
> > >>> On Thu, Mar 17, 2016 at 4:50 PM, Jean-Baptiste Onofré <
> jb@nanthrax.net
> > >
> > >>> wrote:
> > >>>
> > >>>> Great job !
> > >>>>
> > >>>> By the way, when you use ~ in the matrix, does it mean that it works
> > only
> > >>>> in some cases (depending of the pipeline or transform) or it doesn't
> > work
> > >>>> as expected ? Just curious for the Aggregators and the meaning in
> the
> > >>>> Beam
> > >>>> Model.
> > >>>>
> > >>>> Thanks,
> > >>>> Regards
> > >>>> JB
> > >>>>
> > >>>>
> > >>>> On 03/18/2016 12:45 AM, Tyler Akidau wrote:
> > >>>>
> > >>>>> Just pushed the capability matrix and an attendant blog post to the
> > >>>>> site:
> > >>>>>
> > >>>>> - Blog post:
> > >>>>>
> > >>>>>
> > >>>>>
> >
> http://beam.incubator.apache.org/beam/capability/2016/03/17/capability-matrix.html
> > >>>>> - Matrix: http://beam.incubator.apache.org/capability-matrix/
> > >>>>>
> > >>>>> For those of you that want to keep the matrix up to date as your
> > runner
> > >>>>> evolves, you'll want to make updates in the
> > _data/capability-matrix.yml
> > >>>>> file:
> > >>>>>
> > >>>>>
> > >>>>>
> >
> https://github.com/apache/incubator-beam-site/blob/asf-site/_data/capability-matrix.yml
> > >>>>>
> > >>>>> Thanks to everyone for helping fill out the initial set of
> > capabilities!
> > >>>>> Looking forward to updates as things progress. :-)
> > >>>>>
> > >>>>> And thanks also to Max for moving all the website stuff to git!
> > >>>>>
> > >>>>> -Tyler
> > >>>>>
> > >>>>>
> > >>>>> On Sat, Mar 12, 2016 at 9:37 AM Tyler Akidau <ta...@google.com>
> > wrote:
> > >>>>>
> > >>>>> Thanks all! At this point, it looks like most all of the fields
> have
> > >>>>> been
> > >>>>>>
> > >>>>>> filled out. I'm in the process of migrating the spreadsheet
> > contents to
> > >>>>>> YAML within the website source, so I've revoked edit access from
> the
> > >>>>>> doc
> > >>>>>> to
> > >>>>>> keep things from changing while I'm doing that. If you have
> further
> > >>>>>> edits
> > >>>>>> to make, feel free to leave a comment, and I'll incorporate it
> into
> > the
> > >>>>>> YAML.
> > >>>>>>
> > >>>>>> -Tyler
> > >>>>>>
> > >>>>>>
> > >>>>>> On Thu, Mar 10, 2016 at 12:43 AM Jean-Baptiste Onofré <
> > jb@nanthrax.net>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>> Hi Tyler,
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> good idea !
> > >>>>>>>
> > >>>>>>> I like it !
> > >>>>>>>
> > >>>>>>> Regards
> > >>>>>>> JB
> > >>>>>>>
> > >>>>>>> On 03/09/2016 11:14 PM, Tyler Akidau wrote:
> > >>>>>>>
> > >>>>>>>> I just filed BEAM-104
> > >>>>>>>> <https://issues.apache.org/jira/browse/BEAM-104>
> > >>>>>>>> regarding publishing a capability matrix on the Beam website.
> > We've
> > >>>>>>>>
> > >>>>>>> seeded
> > >>>>>>>
> > >>>>>>>> the spreadsheet linked there (
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> >
> https://docs.google.com/spreadsheets/d/1OM077lZBARrtUi6g0X0O0PHaIbFKCD6v0djRefQRE1I/edit
> > >>>>>>>
> > >>>>>>>> )
> > >>>>>>>> with an initial proposed set of capabilities, as well as
> > descriptions
> > >>>>>>>>
> > >>>>>>> for
> > >>>>>>>
> > >>>>>>>> the model and Cloud Dataflow. If folks for other runners
> > (currently
> > >>>>>>>>
> > >>>>>>> Flink
> > >>>>>>>
> > >>>>>>>> and Spark) could please make sure their columns are filled out
> as
> > >>>>>>>> well,
> > >>>>>>>> it'd be much appreciated. Also let us know if there are
> > capabilities
> > >>>>>>>> you
> > >>>>>>>> think we've missed.
> > >>>>>>>>
> > >>>>>>>> Our hope is to get this up and published soon, since we've been
> > >>>>>>>> getting
> > >>>>>>>>
> > >>>>>>> a
> > >>>>>>>
> > >>>>>>>> lot of questions regarding runner capabilities, portability,
> etc.
> > >>>>>>>>
> > >>>>>>>> -Tyler
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>> --
> > >>>>>>> Jean-Baptiste Onofré
> > >>>>>>> jbonofre@apache.org
> > >>>>>>> http://blog.nanthrax.net
> > >>>>>>> Talend - http://www.talend.com
> > >>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>> --
> > >>>> Jean-Baptiste Onofré
> > >>>> jbonofre@apache.org
> > >>>> http://blog.nanthrax.net
> > >>>> Talend - http://www.talend.com
> > >>>>
> > >>>
> > >>
> > >> --
> > >> Jean-Baptiste Onofré
> > >> jbonofre@apache.org
> > >> http://blog.nanthrax.net
> > >> Talend - http://www.talend.com
> >
> >
>
Re: Capability Matrix
Posted by Amit Sela <am...@gmail.com>.
Looks great!
I think it's the best way to give a clear picture of capabilities for users
and runner developers.
And as always, Love the colours ;)
On Fri, Mar 18, 2016 at 3:33 PM Kostas Kloudas <k....@data-artisans.com>
wrote:
> Great to have an overview of the available
> runners and a comprehensible visualization of
> the features each one supports!
>
> Kostas
>
> > On Mar 18, 2016, at 11:32 AM, Maximilian Michels <mx...@apache.org> wrote:
> >
> > Well done. The matrix provides a good basis for improving the existing
> > runners. Moreover, new backends can use it to evaluate capabilities
> > for creating a runner.
> >
> > On Fri, Mar 18, 2016 at 1:15 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
> >> Catcha, thanks !
> >>
> >> Regards
> >> JB
> >>
> >>
> >> On 03/18/2016 12:51 AM, Frances Perry wrote:
> >>>
> >>> That's "partially". Check out the full matrix for complete details:
> >>> http://beam.incubator.apache.org/capability-matrix/
> >>>
> >>> On Thu, Mar 17, 2016 at 4:50 PM, Jean-Baptiste Onofré <jb@nanthrax.net
> >
> >>> wrote:
> >>>
> >>>> Great job !
> >>>>
> >>>> By the way, when you use ~ in the matrix, does it mean that it works
> only
> >>>> in some cases (depending of the pipeline or transform) or it doesn't
> work
> >>>> as expected ? Just curious for the Aggregators and the meaning in the
> >>>> Beam
> >>>> Model.
> >>>>
> >>>> Thanks,
> >>>> Regards
> >>>> JB
> >>>>
> >>>>
> >>>> On 03/18/2016 12:45 AM, Tyler Akidau wrote:
> >>>>
> >>>>> Just pushed the capability matrix and an attendant blog post to the
> >>>>> site:
> >>>>>
> >>>>> - Blog post:
> >>>>>
> >>>>>
> >>>>>
> http://beam.incubator.apache.org/beam/capability/2016/03/17/capability-matrix.html
> >>>>> - Matrix: http://beam.incubator.apache.org/capability-matrix/
> >>>>>
> >>>>> For those of you that want to keep the matrix up to date as your
> runner
> >>>>> evolves, you'll want to make updates in the
> _data/capability-matrix.yml
> >>>>> file:
> >>>>>
> >>>>>
> >>>>>
> https://github.com/apache/incubator-beam-site/blob/asf-site/_data/capability-matrix.yml
> >>>>>
> >>>>> Thanks to everyone for helping fill out the initial set of
> capabilities!
> >>>>> Looking forward to updates as things progress. :-)
> >>>>>
> >>>>> And thanks also to Max for moving all the website stuff to git!
> >>>>>
> >>>>> -Tyler
> >>>>>
> >>>>>
> >>>>> On Sat, Mar 12, 2016 at 9:37 AM Tyler Akidau <ta...@google.com>
> wrote:
> >>>>>
> >>>>> Thanks all! At this point, it looks like most all of the fields have
> >>>>> been
> >>>>>>
> >>>>>> filled out. I'm in the process of migrating the spreadsheet
> contents to
> >>>>>> YAML within the website source, so I've revoked edit access from the
> >>>>>> doc
> >>>>>> to
> >>>>>> keep things from changing while I'm doing that. If you have further
> >>>>>> edits
> >>>>>> to make, feel free to leave a comment, and I'll incorporate it into
> the
> >>>>>> YAML.
> >>>>>>
> >>>>>> -Tyler
> >>>>>>
> >>>>>>
> >>>>>> On Thu, Mar 10, 2016 at 12:43 AM Jean-Baptiste Onofré <
> jb@nanthrax.net>
> >>>>>> wrote:
> >>>>>>
> >>>>>> Hi Tyler,
> >>>>>>>
> >>>>>>>
> >>>>>>> good idea !
> >>>>>>>
> >>>>>>> I like it !
> >>>>>>>
> >>>>>>> Regards
> >>>>>>> JB
> >>>>>>>
> >>>>>>> On 03/09/2016 11:14 PM, Tyler Akidau wrote:
> >>>>>>>
> >>>>>>>> I just filed BEAM-104
> >>>>>>>> <https://issues.apache.org/jira/browse/BEAM-104>
> >>>>>>>> regarding publishing a capability matrix on the Beam website.
> We've
> >>>>>>>>
> >>>>>>> seeded
> >>>>>>>
> >>>>>>>> the spreadsheet linked there (
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> https://docs.google.com/spreadsheets/d/1OM077lZBARrtUi6g0X0O0PHaIbFKCD6v0djRefQRE1I/edit
> >>>>>>>
> >>>>>>>> )
> >>>>>>>> with an initial proposed set of capabilities, as well as
> descriptions
> >>>>>>>>
> >>>>>>> for
> >>>>>>>
> >>>>>>>> the model and Cloud Dataflow. If folks for other runners
> (currently
> >>>>>>>>
> >>>>>>> Flink
> >>>>>>>
> >>>>>>>> and Spark) could please make sure their columns are filled out as
> >>>>>>>> well,
> >>>>>>>> it'd be much appreciated. Also let us know if there are
> capabilities
> >>>>>>>> you
> >>>>>>>> think we've missed.
> >>>>>>>>
> >>>>>>>> Our hope is to get this up and published soon, since we've been
> >>>>>>>> getting
> >>>>>>>>
> >>>>>>> a
> >>>>>>>
> >>>>>>>> lot of questions regarding runner capabilities, portability, etc.
> >>>>>>>>
> >>>>>>>> -Tyler
> >>>>>>>>
> >>>>>>>>
> >>>>>>> --
> >>>>>>> Jean-Baptiste Onofré
> >>>>>>> jbonofre@apache.org
> >>>>>>> http://blog.nanthrax.net
> >>>>>>> Talend - http://www.talend.com
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>> --
> >>>> Jean-Baptiste Onofré
> >>>> jbonofre@apache.org
> >>>> http://blog.nanthrax.net
> >>>> Talend - http://www.talend.com
> >>>>
> >>>
> >>
> >> --
> >> Jean-Baptiste Onofré
> >> jbonofre@apache.org
> >> http://blog.nanthrax.net
> >> Talend - http://www.talend.com
>
>
Re: Capability Matrix
Posted by Kostas Kloudas <k....@data-artisans.com>.
Great to have an overview of the available
runners and a comprehensible visualization of
the features each one supports!
Kostas
> On Mar 18, 2016, at 11:32 AM, Maximilian Michels <mx...@apache.org> wrote:
>
> Well done. The matrix provides a good basis for improving the existing
> runners. Moreover, new backends can use it to evaluate capabilities
> for creating a runner.
>
> On Fri, Mar 18, 2016 at 1:15 AM, Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>> Catcha, thanks !
>>
>> Regards
>> JB
>>
>>
>> On 03/18/2016 12:51 AM, Frances Perry wrote:
>>>
>>> That's "partially". Check out the full matrix for complete details:
>>> http://beam.incubator.apache.org/capability-matrix/
>>>
>>> On Thu, Mar 17, 2016 at 4:50 PM, Jean-Baptiste Onofré <jb...@nanthrax.net>
>>> wrote:
>>>
>>>> Great job !
>>>>
>>>> By the way, when you use ~ in the matrix, does it mean that it works only
>>>> in some cases (depending of the pipeline or transform) or it doesn't work
>>>> as expected ? Just curious for the Aggregators and the meaning in the
>>>> Beam
>>>> Model.
>>>>
>>>> Thanks,
>>>> Regards
>>>> JB
>>>>
>>>>
>>>> On 03/18/2016 12:45 AM, Tyler Akidau wrote:
>>>>
>>>>> Just pushed the capability matrix and an attendant blog post to the
>>>>> site:
>>>>>
>>>>> - Blog post:
>>>>>
>>>>>
>>>>> http://beam.incubator.apache.org/beam/capability/2016/03/17/capability-matrix.html
>>>>> - Matrix: http://beam.incubator.apache.org/capability-matrix/
>>>>>
>>>>> For those of you that want to keep the matrix up to date as your runner
>>>>> evolves, you'll want to make updates in the _data/capability-matrix.yml
>>>>> file:
>>>>>
>>>>>
>>>>> https://github.com/apache/incubator-beam-site/blob/asf-site/_data/capability-matrix.yml
>>>>>
>>>>> Thanks to everyone for helping fill out the initial set of capabilities!
>>>>> Looking forward to updates as things progress. :-)
>>>>>
>>>>> And thanks also to Max for moving all the website stuff to git!
>>>>>
>>>>> -Tyler
>>>>>
>>>>>
>>>>> On Sat, Mar 12, 2016 at 9:37 AM Tyler Akidau <ta...@google.com> wrote:
>>>>>
>>>>> Thanks all! At this point, it looks like most all of the fields have
>>>>> been
>>>>>>
>>>>>> filled out. I'm in the process of migrating the spreadsheet contents to
>>>>>> YAML within the website source, so I've revoked edit access from the
>>>>>> doc
>>>>>> to
>>>>>> keep things from changing while I'm doing that. If you have further
>>>>>> edits
>>>>>> to make, feel free to leave a comment, and I'll incorporate it into the
>>>>>> YAML.
>>>>>>
>>>>>> -Tyler
>>>>>>
>>>>>>
>>>>>> On Thu, Mar 10, 2016 at 12:43 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>>>> wrote:
>>>>>>
>>>>>> Hi Tyler,
>>>>>>>
>>>>>>>
>>>>>>> good idea !
>>>>>>>
>>>>>>> I like it !
>>>>>>>
>>>>>>> Regards
>>>>>>> JB
>>>>>>>
>>>>>>> On 03/09/2016 11:14 PM, Tyler Akidau wrote:
>>>>>>>
>>>>>>>> I just filed BEAM-104
>>>>>>>> <https://issues.apache.org/jira/browse/BEAM-104>
>>>>>>>> regarding publishing a capability matrix on the Beam website. We've
>>>>>>>>
>>>>>>> seeded
>>>>>>>
>>>>>>>> the spreadsheet linked there (
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> https://docs.google.com/spreadsheets/d/1OM077lZBARrtUi6g0X0O0PHaIbFKCD6v0djRefQRE1I/edit
>>>>>>>
>>>>>>>> )
>>>>>>>> with an initial proposed set of capabilities, as well as descriptions
>>>>>>>>
>>>>>>> for
>>>>>>>
>>>>>>>> the model and Cloud Dataflow. If folks for other runners (currently
>>>>>>>>
>>>>>>> Flink
>>>>>>>
>>>>>>>> and Spark) could please make sure their columns are filled out as
>>>>>>>> well,
>>>>>>>> it'd be much appreciated. Also let us know if there are capabilities
>>>>>>>> you
>>>>>>>> think we've missed.
>>>>>>>>
>>>>>>>> Our hope is to get this up and published soon, since we've been
>>>>>>>> getting
>>>>>>>>
>>>>>>> a
>>>>>>>
>>>>>>>> lot of questions regarding runner capabilities, portability, etc.
>>>>>>>>
>>>>>>>> -Tyler
>>>>>>>>
>>>>>>>>
>>>>>>> --
>>>>>>> Jean-Baptiste Onofré
>>>>>>> jbonofre@apache.org
>>>>>>> http://blog.nanthrax.net
>>>>>>> Talend - http://www.talend.com
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>> --
>>>> Jean-Baptiste Onofré
>>>> jbonofre@apache.org
>>>> http://blog.nanthrax.net
>>>> Talend - http://www.talend.com
>>>>
>>>
>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
Re: Capability Matrix
Posted by Maximilian Michels <mx...@apache.org>.
Well done. The matrix provides a good basis for improving the existing
runners. Moreover, new backends can use it to evaluate capabilities
for creating a runner.
On Fri, Mar 18, 2016 at 1:15 AM, Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
> Catcha, thanks !
>
> Regards
> JB
>
>
> On 03/18/2016 12:51 AM, Frances Perry wrote:
>>
>> That's "partially". Check out the full matrix for complete details:
>> http://beam.incubator.apache.org/capability-matrix/
>>
>> On Thu, Mar 17, 2016 at 4:50 PM, Jean-Baptiste Onofré <jb...@nanthrax.net>
>> wrote:
>>
>>> Great job !
>>>
>>> By the way, when you use ~ in the matrix, does it mean that it works only
>>> in some cases (depending of the pipeline or transform) or it doesn't work
>>> as expected ? Just curious for the Aggregators and the meaning in the
>>> Beam
>>> Model.
>>>
>>> Thanks,
>>> Regards
>>> JB
>>>
>>>
>>> On 03/18/2016 12:45 AM, Tyler Akidau wrote:
>>>
>>>> Just pushed the capability matrix and an attendant blog post to the
>>>> site:
>>>>
>>>> - Blog post:
>>>>
>>>>
>>>> http://beam.incubator.apache.org/beam/capability/2016/03/17/capability-matrix.html
>>>> - Matrix: http://beam.incubator.apache.org/capability-matrix/
>>>>
>>>> For those of you that want to keep the matrix up to date as your runner
>>>> evolves, you'll want to make updates in the _data/capability-matrix.yml
>>>> file:
>>>>
>>>>
>>>> https://github.com/apache/incubator-beam-site/blob/asf-site/_data/capability-matrix.yml
>>>>
>>>> Thanks to everyone for helping fill out the initial set of capabilities!
>>>> Looking forward to updates as things progress. :-)
>>>>
>>>> And thanks also to Max for moving all the website stuff to git!
>>>>
>>>> -Tyler
>>>>
>>>>
>>>> On Sat, Mar 12, 2016 at 9:37 AM Tyler Akidau <ta...@google.com> wrote:
>>>>
>>>> Thanks all! At this point, it looks like most all of the fields have
>>>> been
>>>>>
>>>>> filled out. I'm in the process of migrating the spreadsheet contents to
>>>>> YAML within the website source, so I've revoked edit access from the
>>>>> doc
>>>>> to
>>>>> keep things from changing while I'm doing that. If you have further
>>>>> edits
>>>>> to make, feel free to leave a comment, and I'll incorporate it into the
>>>>> YAML.
>>>>>
>>>>> -Tyler
>>>>>
>>>>>
>>>>> On Thu, Mar 10, 2016 at 12:43 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>>> wrote:
>>>>>
>>>>> Hi Tyler,
>>>>>>
>>>>>>
>>>>>> good idea !
>>>>>>
>>>>>> I like it !
>>>>>>
>>>>>> Regards
>>>>>> JB
>>>>>>
>>>>>> On 03/09/2016 11:14 PM, Tyler Akidau wrote:
>>>>>>
>>>>>>> I just filed BEAM-104
>>>>>>> <https://issues.apache.org/jira/browse/BEAM-104>
>>>>>>> regarding publishing a capability matrix on the Beam website. We've
>>>>>>>
>>>>>> seeded
>>>>>>
>>>>>>> the spreadsheet linked there (
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> https://docs.google.com/spreadsheets/d/1OM077lZBARrtUi6g0X0O0PHaIbFKCD6v0djRefQRE1I/edit
>>>>>>
>>>>>>> )
>>>>>>> with an initial proposed set of capabilities, as well as descriptions
>>>>>>>
>>>>>> for
>>>>>>
>>>>>>> the model and Cloud Dataflow. If folks for other runners (currently
>>>>>>>
>>>>>> Flink
>>>>>>
>>>>>>> and Spark) could please make sure their columns are filled out as
>>>>>>> well,
>>>>>>> it'd be much appreciated. Also let us know if there are capabilities
>>>>>>> you
>>>>>>> think we've missed.
>>>>>>>
>>>>>>> Our hope is to get this up and published soon, since we've been
>>>>>>> getting
>>>>>>>
>>>>>> a
>>>>>>
>>>>>>> lot of questions regarding runner capabilities, portability, etc.
>>>>>>>
>>>>>>> -Tyler
>>>>>>>
>>>>>>>
>>>>>> --
>>>>>> Jean-Baptiste Onofré
>>>>>> jbonofre@apache.org
>>>>>> http://blog.nanthrax.net
>>>>>> Talend - http://www.talend.com
>>>>>>
>>>>>>
>>>>>
>>>>
>>> --
>>> Jean-Baptiste Onofré
>>> jbonofre@apache.org
>>> http://blog.nanthrax.net
>>> Talend - http://www.talend.com
>>>
>>
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
Re: Capability Matrix
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Catcha, thanks !
Regards
JB
On 03/18/2016 12:51 AM, Frances Perry wrote:
> That's "partially". Check out the full matrix for complete details:
> http://beam.incubator.apache.org/capability-matrix/
>
> On Thu, Mar 17, 2016 at 4:50 PM, Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
>> Great job !
>>
>> By the way, when you use ~ in the matrix, does it mean that it works only
>> in some cases (depending of the pipeline or transform) or it doesn't work
>> as expected ? Just curious for the Aggregators and the meaning in the Beam
>> Model.
>>
>> Thanks,
>> Regards
>> JB
>>
>>
>> On 03/18/2016 12:45 AM, Tyler Akidau wrote:
>>
>>> Just pushed the capability matrix and an attendant blog post to the site:
>>>
>>> - Blog post:
>>>
>>> http://beam.incubator.apache.org/beam/capability/2016/03/17/capability-matrix.html
>>> - Matrix: http://beam.incubator.apache.org/capability-matrix/
>>>
>>> For those of you that want to keep the matrix up to date as your runner
>>> evolves, you'll want to make updates in the _data/capability-matrix.yml
>>> file:
>>>
>>> https://github.com/apache/incubator-beam-site/blob/asf-site/_data/capability-matrix.yml
>>>
>>> Thanks to everyone for helping fill out the initial set of capabilities!
>>> Looking forward to updates as things progress. :-)
>>>
>>> And thanks also to Max for moving all the website stuff to git!
>>>
>>> -Tyler
>>>
>>>
>>> On Sat, Mar 12, 2016 at 9:37 AM Tyler Akidau <ta...@google.com> wrote:
>>>
>>> Thanks all! At this point, it looks like most all of the fields have been
>>>> filled out. I'm in the process of migrating the spreadsheet contents to
>>>> YAML within the website source, so I've revoked edit access from the doc
>>>> to
>>>> keep things from changing while I'm doing that. If you have further edits
>>>> to make, feel free to leave a comment, and I'll incorporate it into the
>>>> YAML.
>>>>
>>>> -Tyler
>>>>
>>>>
>>>> On Thu, Mar 10, 2016 at 12:43 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>> wrote:
>>>>
>>>> Hi Tyler,
>>>>>
>>>>> good idea !
>>>>>
>>>>> I like it !
>>>>>
>>>>> Regards
>>>>> JB
>>>>>
>>>>> On 03/09/2016 11:14 PM, Tyler Akidau wrote:
>>>>>
>>>>>> I just filed BEAM-104 <https://issues.apache.org/jira/browse/BEAM-104>
>>>>>> regarding publishing a capability matrix on the Beam website. We've
>>>>>>
>>>>> seeded
>>>>>
>>>>>> the spreadsheet linked there (
>>>>>>
>>>>>>
>>>>> https://docs.google.com/spreadsheets/d/1OM077lZBARrtUi6g0X0O0PHaIbFKCD6v0djRefQRE1I/edit
>>>>>
>>>>>> )
>>>>>> with an initial proposed set of capabilities, as well as descriptions
>>>>>>
>>>>> for
>>>>>
>>>>>> the model and Cloud Dataflow. If folks for other runners (currently
>>>>>>
>>>>> Flink
>>>>>
>>>>>> and Spark) could please make sure their columns are filled out as well,
>>>>>> it'd be much appreciated. Also let us know if there are capabilities
>>>>>> you
>>>>>> think we've missed.
>>>>>>
>>>>>> Our hope is to get this up and published soon, since we've been getting
>>>>>>
>>>>> a
>>>>>
>>>>>> lot of questions regarding runner capabilities, portability, etc.
>>>>>>
>>>>>> -Tyler
>>>>>>
>>>>>>
>>>>> --
>>>>> Jean-Baptiste Onofré
>>>>> jbonofre@apache.org
>>>>> http://blog.nanthrax.net
>>>>> Talend - http://www.talend.com
>>>>>
>>>>>
>>>>
>>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Capability Matrix
Posted by Frances Perry <fj...@google.com.INVALID>.
That's "partially". Check out the full matrix for complete details:
http://beam.incubator.apache.org/capability-matrix/
On Thu, Mar 17, 2016 at 4:50 PM, Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Great job !
>
> By the way, when you use ~ in the matrix, does it mean that it works only
> in some cases (depending of the pipeline or transform) or it doesn't work
> as expected ? Just curious for the Aggregators and the meaning in the Beam
> Model.
>
> Thanks,
> Regards
> JB
>
>
> On 03/18/2016 12:45 AM, Tyler Akidau wrote:
>
>> Just pushed the capability matrix and an attendant blog post to the site:
>>
>> - Blog post:
>>
>> http://beam.incubator.apache.org/beam/capability/2016/03/17/capability-matrix.html
>> - Matrix: http://beam.incubator.apache.org/capability-matrix/
>>
>> For those of you that want to keep the matrix up to date as your runner
>> evolves, you'll want to make updates in the _data/capability-matrix.yml
>> file:
>>
>> https://github.com/apache/incubator-beam-site/blob/asf-site/_data/capability-matrix.yml
>>
>> Thanks to everyone for helping fill out the initial set of capabilities!
>> Looking forward to updates as things progress. :-)
>>
>> And thanks also to Max for moving all the website stuff to git!
>>
>> -Tyler
>>
>>
>> On Sat, Mar 12, 2016 at 9:37 AM Tyler Akidau <ta...@google.com> wrote:
>>
>> Thanks all! At this point, it looks like most all of the fields have been
>>> filled out. I'm in the process of migrating the spreadsheet contents to
>>> YAML within the website source, so I've revoked edit access from the doc
>>> to
>>> keep things from changing while I'm doing that. If you have further edits
>>> to make, feel free to leave a comment, and I'll incorporate it into the
>>> YAML.
>>>
>>> -Tyler
>>>
>>>
>>> On Thu, Mar 10, 2016 at 12:43 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
>>> wrote:
>>>
>>> Hi Tyler,
>>>>
>>>> good idea !
>>>>
>>>> I like it !
>>>>
>>>> Regards
>>>> JB
>>>>
>>>> On 03/09/2016 11:14 PM, Tyler Akidau wrote:
>>>>
>>>>> I just filed BEAM-104 <https://issues.apache.org/jira/browse/BEAM-104>
>>>>> regarding publishing a capability matrix on the Beam website. We've
>>>>>
>>>> seeded
>>>>
>>>>> the spreadsheet linked there (
>>>>>
>>>>>
>>>> https://docs.google.com/spreadsheets/d/1OM077lZBARrtUi6g0X0O0PHaIbFKCD6v0djRefQRE1I/edit
>>>>
>>>>> )
>>>>> with an initial proposed set of capabilities, as well as descriptions
>>>>>
>>>> for
>>>>
>>>>> the model and Cloud Dataflow. If folks for other runners (currently
>>>>>
>>>> Flink
>>>>
>>>>> and Spark) could please make sure their columns are filled out as well,
>>>>> it'd be much appreciated. Also let us know if there are capabilities
>>>>> you
>>>>> think we've missed.
>>>>>
>>>>> Our hope is to get this up and published soon, since we've been getting
>>>>>
>>>> a
>>>>
>>>>> lot of questions regarding runner capabilities, portability, etc.
>>>>>
>>>>> -Tyler
>>>>>
>>>>>
>>>> --
>>>> Jean-Baptiste Onofré
>>>> jbonofre@apache.org
>>>> http://blog.nanthrax.net
>>>> Talend - http://www.talend.com
>>>>
>>>>
>>>
>>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: Capability Matrix
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Great job !
By the way, when you use ~ in the matrix, does it mean that it works
only in some cases (depending of the pipeline or transform) or it
doesn't work as expected ? Just curious for the Aggregators and the
meaning in the Beam Model.
Thanks,
Regards
JB
On 03/18/2016 12:45 AM, Tyler Akidau wrote:
> Just pushed the capability matrix and an attendant blog post to the site:
>
> - Blog post:
> http://beam.incubator.apache.org/beam/capability/2016/03/17/capability-matrix.html
> - Matrix: http://beam.incubator.apache.org/capability-matrix/
>
> For those of you that want to keep the matrix up to date as your runner
> evolves, you'll want to make updates in the _data/capability-matrix.yml
> file:
> https://github.com/apache/incubator-beam-site/blob/asf-site/_data/capability-matrix.yml
>
> Thanks to everyone for helping fill out the initial set of capabilities!
> Looking forward to updates as things progress. :-)
>
> And thanks also to Max for moving all the website stuff to git!
>
> -Tyler
>
>
> On Sat, Mar 12, 2016 at 9:37 AM Tyler Akidau <ta...@google.com> wrote:
>
>> Thanks all! At this point, it looks like most all of the fields have been
>> filled out. I'm in the process of migrating the spreadsheet contents to
>> YAML within the website source, so I've revoked edit access from the doc to
>> keep things from changing while I'm doing that. If you have further edits
>> to make, feel free to leave a comment, and I'll incorporate it into the
>> YAML.
>>
>> -Tyler
>>
>>
>> On Thu, Mar 10, 2016 at 12:43 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
>> wrote:
>>
>>> Hi Tyler,
>>>
>>> good idea !
>>>
>>> I like it !
>>>
>>> Regards
>>> JB
>>>
>>> On 03/09/2016 11:14 PM, Tyler Akidau wrote:
>>>> I just filed BEAM-104 <https://issues.apache.org/jira/browse/BEAM-104>
>>>> regarding publishing a capability matrix on the Beam website. We've
>>> seeded
>>>> the spreadsheet linked there (
>>>>
>>> https://docs.google.com/spreadsheets/d/1OM077lZBARrtUi6g0X0O0PHaIbFKCD6v0djRefQRE1I/edit
>>>> )
>>>> with an initial proposed set of capabilities, as well as descriptions
>>> for
>>>> the model and Cloud Dataflow. If folks for other runners (currently
>>> Flink
>>>> and Spark) could please make sure their columns are filled out as well,
>>>> it'd be much appreciated. Also let us know if there are capabilities you
>>>> think we've missed.
>>>>
>>>> Our hope is to get this up and published soon, since we've been getting
>>> a
>>>> lot of questions regarding runner capabilities, portability, etc.
>>>>
>>>> -Tyler
>>>>
>>>
>>> --
>>> Jean-Baptiste Onofré
>>> jbonofre@apache.org
>>> http://blog.nanthrax.net
>>> Talend - http://www.talend.com
>>>
>>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Capability Matrix
Posted by Tyler Akidau <ta...@google.com.INVALID>.
Just pushed the capability matrix and an attendant blog post to the site:
- Blog post:
http://beam.incubator.apache.org/beam/capability/2016/03/17/capability-matrix.html
- Matrix: http://beam.incubator.apache.org/capability-matrix/
For those of you that want to keep the matrix up to date as your runner
evolves, you'll want to make updates in the _data/capability-matrix.yml
file:
https://github.com/apache/incubator-beam-site/blob/asf-site/_data/capability-matrix.yml
Thanks to everyone for helping fill out the initial set of capabilities!
Looking forward to updates as things progress. :-)
And thanks also to Max for moving all the website stuff to git!
-Tyler
On Sat, Mar 12, 2016 at 9:37 AM Tyler Akidau <ta...@google.com> wrote:
> Thanks all! At this point, it looks like most all of the fields have been
> filled out. I'm in the process of migrating the spreadsheet contents to
> YAML within the website source, so I've revoked edit access from the doc to
> keep things from changing while I'm doing that. If you have further edits
> to make, feel free to leave a comment, and I'll incorporate it into the
> YAML.
>
> -Tyler
>
>
> On Thu, Mar 10, 2016 at 12:43 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
>> Hi Tyler,
>>
>> good idea !
>>
>> I like it !
>>
>> Regards
>> JB
>>
>> On 03/09/2016 11:14 PM, Tyler Akidau wrote:
>> > I just filed BEAM-104 <https://issues.apache.org/jira/browse/BEAM-104>
>> > regarding publishing a capability matrix on the Beam website. We've
>> seeded
>> > the spreadsheet linked there (
>> >
>> https://docs.google.com/spreadsheets/d/1OM077lZBARrtUi6g0X0O0PHaIbFKCD6v0djRefQRE1I/edit
>> > )
>> > with an initial proposed set of capabilities, as well as descriptions
>> for
>> > the model and Cloud Dataflow. If folks for other runners (currently
>> Flink
>> > and Spark) could please make sure their columns are filled out as well,
>> > it'd be much appreciated. Also let us know if there are capabilities you
>> > think we've missed.
>> >
>> > Our hope is to get this up and published soon, since we've been getting
>> a
>> > lot of questions regarding runner capabilities, portability, etc.
>> >
>> > -Tyler
>> >
>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>
Re: Capability Matrix
Posted by Tyler Akidau <ta...@google.com.INVALID>.
Thanks all! At this point, it looks like most all of the fields have been
filled out. I'm in the process of migrating the spreadsheet contents to
YAML within the website source, so I've revoked edit access from the doc to
keep things from changing while I'm doing that. If you have further edits
to make, feel free to leave a comment, and I'll incorporate it into the
YAML.
-Tyler
On Thu, Mar 10, 2016 at 12:43 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Hi Tyler,
>
> good idea !
>
> I like it !
>
> Regards
> JB
>
> On 03/09/2016 11:14 PM, Tyler Akidau wrote:
> > I just filed BEAM-104 <https://issues.apache.org/jira/browse/BEAM-104>
> > regarding publishing a capability matrix on the Beam website. We've
> seeded
> > the spreadsheet linked there (
> >
> https://docs.google.com/spreadsheets/d/1OM077lZBARrtUi6g0X0O0PHaIbFKCD6v0djRefQRE1I/edit
> > )
> > with an initial proposed set of capabilities, as well as descriptions for
> > the model and Cloud Dataflow. If folks for other runners (currently Flink
> > and Spark) could please make sure their columns are filled out as well,
> > it'd be much appreciated. Also let us know if there are capabilities you
> > think we've missed.
> >
> > Our hope is to get this up and published soon, since we've been getting a
> > lot of questions regarding runner capabilities, portability, etc.
> >
> > -Tyler
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: Capability Matrix
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Tyler,
good idea !
I like it !
Regards
JB
On 03/09/2016 11:14 PM, Tyler Akidau wrote:
> I just filed BEAM-104 <https://issues.apache.org/jira/browse/BEAM-104>
> regarding publishing a capability matrix on the Beam website. We've seeded
> the spreadsheet linked there (
> https://docs.google.com/spreadsheets/d/1OM077lZBARrtUi6g0X0O0PHaIbFKCD6v0djRefQRE1I/edit
> )
> with an initial proposed set of capabilities, as well as descriptions for
> the model and Cloud Dataflow. If folks for other runners (currently Flink
> and Spark) could please make sure their columns are filled out as well,
> it'd be much appreciated. Also let us know if there are capabilities you
> think we've missed.
>
> Our hope is to get this up and published soon, since we've been getting a
> lot of questions regarding runner capabilities, portability, etc.
>
> -Tyler
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Capability Matrix
Posted by Kostas Tzoumas <kt...@apache.org>.
Great initiative!
Together with Kostas and Max we made a first pass over the matrix for the
Flink runner capabilities.
On Wed, Mar 9, 2016 at 11:14 PM, Tyler Akidau <ta...@google.com.invalid>
wrote:
> I just filed BEAM-104 <https://issues.apache.org/jira/browse/BEAM-104>
> regarding publishing a capability matrix on the Beam website. We've seeded
> the spreadsheet linked there (
>
> https://docs.google.com/spreadsheets/d/1OM077lZBARrtUi6g0X0O0PHaIbFKCD6v0djRefQRE1I/edit
> )
> with an initial proposed set of capabilities, as well as descriptions for
> the model and Cloud Dataflow. If folks for other runners (currently Flink
> and Spark) could please make sure their columns are filled out as well,
> it'd be much appreciated. Also let us know if there are capabilities you
> think we've missed.
>
> Our hope is to get this up and published soon, since we've been getting a
> lot of questions regarding runner capabilities, portability, etc.
>
> -Tyler
>