You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Amit Sela <am...@apache.org> on 2016/02/10 19:46:37 UTC
Spark Runner Technical Vision
Following the technical vision Frances sent, and following the previous
document I wrote on the spark runner gaps, I'm attaching a suggested
technical vision to the Spark runner.
I tried to be inline with the project's technical vision both in timeline,
and in document style. I suggest that once the ASF Wiki is up, we'll have
roadmap for runners as well.
Thanks,
Amit
https://docs.google.com/document/d/1y4qlQinjjrusGWlgq-mYmbxRW2z7-_X5Xax-GG0YsC0/edit?usp=sharing
Re: Spark Runner Technical Vision
Posted by Amit Sela <am...@gmail.com>.
Thanks Frances!
On Thu, Feb 11, 2016 at 3:25 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Good idea Tyler.
>
> Regards
> JB
>
> On 02/11/2016 07:55 AM, Tyler Akidau wrote:
> > +1
> >
> > Although, should/could we have a shared folder to store things in as well
> > (since we probably don't want to be linking everything from the website,
> > but it'd be nice to have a consistent place for all relevant docs to be
> > found)?
> >
> > -Tyler
> >
> > On Wed, Feb 10, 2016 at 10:51 PM James Malone
> > <ja...@google.com.invalid> wrote:
> >
> >> Any objections if I link to this (and other design) doc on the
> (upcoming)
> >> project site (beam.incubator.apache.org)?
> >>
> >> On Wed, Feb 10, 2016 at 10:31 PM, Tyler Akidau
> <takidau@google.com.invalid
> >>>
> >> wrote:
> >>
> >>> +1. Added a bunch of comments. Thanks!
> >>>
> >>> On Wed, Feb 10, 2016 at 9:35 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
> >>> wrote:
> >>>
> >>>> Thanks Amit.
> >>>>
> >>>> Regards
> >>>> JB
> >>>>
> >>>> On 02/10/2016 07:46 PM, Amit Sela wrote:
> >>>>> Following the technical vision Frances sent, and following the
> >> previous
> >>>>> document I wrote on the spark runner gaps, I'm attaching a suggested
> >>>>> technical vision to the Spark runner.
> >>>>>
> >>>>> I tried to be inline with the project's technical vision both in
> >>>> timeline,
> >>>>> and in document style. I suggest that once the ASF Wiki is up, we'll
> >>> have
> >>>>> roadmap for runners as well.
> >>>>>
> >>>>> Thanks,
> >>>>> Amit
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
> https://docs.google.com/document/d/1y4qlQinjjrusGWlgq-mYmbxRW2z7-_X5Xax-GG0YsC0/edit?usp=sharing
> >>>>>
> >>>>
> >>>> --
> >>>> Jean-Baptiste Onofré
> >>>> jbonofre@apache.org
> >>>> http://blog.nanthrax.net
> >>>> Talend - http://www.talend.com
> >>>>
> >>>
> >>
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: Spark Runner Technical Vision
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Good idea Tyler.
Regards
JB
On 02/11/2016 07:55 AM, Tyler Akidau wrote:
> +1
>
> Although, should/could we have a shared folder to store things in as well
> (since we probably don't want to be linking everything from the website,
> but it'd be nice to have a consistent place for all relevant docs to be
> found)?
>
> -Tyler
>
> On Wed, Feb 10, 2016 at 10:51 PM James Malone
> <ja...@google.com.invalid> wrote:
>
>> Any objections if I link to this (and other design) doc on the (upcoming)
>> project site (beam.incubator.apache.org)?
>>
>> On Wed, Feb 10, 2016 at 10:31 PM, Tyler Akidau <takidau@google.com.invalid
>>>
>> wrote:
>>
>>> +1. Added a bunch of comments. Thanks!
>>>
>>> On Wed, Feb 10, 2016 at 9:35 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>>> wrote:
>>>
>>>> Thanks Amit.
>>>>
>>>> Regards
>>>> JB
>>>>
>>>> On 02/10/2016 07:46 PM, Amit Sela wrote:
>>>>> Following the technical vision Frances sent, and following the
>> previous
>>>>> document I wrote on the spark runner gaps, I'm attaching a suggested
>>>>> technical vision to the Spark runner.
>>>>>
>>>>> I tried to be inline with the project's technical vision both in
>>>> timeline,
>>>>> and in document style. I suggest that once the ASF Wiki is up, we'll
>>> have
>>>>> roadmap for runners as well.
>>>>>
>>>>> Thanks,
>>>>> Amit
>>>>>
>>>>>
>>>>
>>>
>> https://docs.google.com/document/d/1y4qlQinjjrusGWlgq-mYmbxRW2z7-_X5Xax-GG0YsC0/edit?usp=sharing
>>>>>
>>>>
>>>> --
>>>> Jean-Baptiste Onofré
>>>> jbonofre@apache.org
>>>> http://blog.nanthrax.net
>>>> Talend - http://www.talend.com
>>>>
>>>
>>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Spark Runner Technical Vision
Posted by James Malone <ja...@google.com.INVALID>.
Excellent! In the draft website I've linked to that folder so everyone can
find it.
On Thu, Feb 11, 2016 at 5:18 AM, Frances Perry <fj...@google.com.invalid>
wrote:
> I've been adding things to this folder:
>
> https://drive.google.com/open?id=0B-IhJZh9Ab52OFBVZHpsNjc4eXc
>
> Should have publicized it ;-)
>
> On Wed, Feb 10, 2016 at 11:20 PM, James Malone <
> jamesmalone@google.com.invalid> wrote:
>
> > Makes complete sense to me. :)
> >
> > On Wed, Feb 10, 2016 at 10:55 PM, Tyler Akidau
> <takidau@google.com.invalid
> > >
> > wrote:
> >
> > > +1
> > >
> > > Although, should/could we have a shared folder to store things in as
> well
> > > (since we probably don't want to be linking everything from the
> website,
> > > but it'd be nice to have a consistent place for all relevant docs to be
> > > found)?
> > >
> > > -Tyler
> > >
> > > On Wed, Feb 10, 2016 at 10:51 PM James Malone
> > > <ja...@google.com.invalid> wrote:
> > >
> > > > Any objections if I link to this (and other design) doc on the
> > (upcoming)
> > > > project site (beam.incubator.apache.org)?
> > > >
> > > > On Wed, Feb 10, 2016 at 10:31 PM, Tyler Akidau
> > > <takidau@google.com.invalid
> > > > >
> > > > wrote:
> > > >
> > > > > +1. Added a bunch of comments. Thanks!
> > > > >
> > > > > On Wed, Feb 10, 2016 at 9:35 PM Jean-Baptiste Onofré <
> > jb@nanthrax.net>
> > > > > wrote:
> > > > >
> > > > > > Thanks Amit.
> > > > > >
> > > > > > Regards
> > > > > > JB
> > > > > >
> > > > > > On 02/10/2016 07:46 PM, Amit Sela wrote:
> > > > > > > Following the technical vision Frances sent, and following the
> > > > previous
> > > > > > > document I wrote on the spark runner gaps, I'm attaching a
> > > suggested
> > > > > > > technical vision to the Spark runner.
> > > > > > >
> > > > > > > I tried to be inline with the project's technical vision both
> in
> > > > > > timeline,
> > > > > > > and in document style. I suggest that once the ASF Wiki is up,
> > > we'll
> > > > > have
> > > > > > > roadmap for runners as well.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Amit
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.google.com/document/d/1y4qlQinjjrusGWlgq-mYmbxRW2z7-_X5Xax-GG0YsC0/edit?usp=sharing
> > > > > > >
> > > > > >
> > > > > > --
> > > > > > Jean-Baptiste Onofré
> > > > > > jbonofre@apache.org
> > > > > > http://blog.nanthrax.net
> > > > > > Talend - http://www.talend.com
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Spark Runner Technical Vision
Posted by Frances Perry <fj...@google.com.INVALID>.
I've been adding things to this folder:
https://drive.google.com/open?id=0B-IhJZh9Ab52OFBVZHpsNjc4eXc
Should have publicized it ;-)
On Wed, Feb 10, 2016 at 11:20 PM, James Malone <
jamesmalone@google.com.invalid> wrote:
> Makes complete sense to me. :)
>
> On Wed, Feb 10, 2016 at 10:55 PM, Tyler Akidau <takidau@google.com.invalid
> >
> wrote:
>
> > +1
> >
> > Although, should/could we have a shared folder to store things in as well
> > (since we probably don't want to be linking everything from the website,
> > but it'd be nice to have a consistent place for all relevant docs to be
> > found)?
> >
> > -Tyler
> >
> > On Wed, Feb 10, 2016 at 10:51 PM James Malone
> > <ja...@google.com.invalid> wrote:
> >
> > > Any objections if I link to this (and other design) doc on the
> (upcoming)
> > > project site (beam.incubator.apache.org)?
> > >
> > > On Wed, Feb 10, 2016 at 10:31 PM, Tyler Akidau
> > <takidau@google.com.invalid
> > > >
> > > wrote:
> > >
> > > > +1. Added a bunch of comments. Thanks!
> > > >
> > > > On Wed, Feb 10, 2016 at 9:35 PM Jean-Baptiste Onofré <
> jb@nanthrax.net>
> > > > wrote:
> > > >
> > > > > Thanks Amit.
> > > > >
> > > > > Regards
> > > > > JB
> > > > >
> > > > > On 02/10/2016 07:46 PM, Amit Sela wrote:
> > > > > > Following the technical vision Frances sent, and following the
> > > previous
> > > > > > document I wrote on the spark runner gaps, I'm attaching a
> > suggested
> > > > > > technical vision to the Spark runner.
> > > > > >
> > > > > > I tried to be inline with the project's technical vision both in
> > > > > timeline,
> > > > > > and in document style. I suggest that once the ASF Wiki is up,
> > we'll
> > > > have
> > > > > > roadmap for runners as well.
> > > > > >
> > > > > > Thanks,
> > > > > > Amit
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.google.com/document/d/1y4qlQinjjrusGWlgq-mYmbxRW2z7-_X5Xax-GG0YsC0/edit?usp=sharing
> > > > > >
> > > > >
> > > > > --
> > > > > Jean-Baptiste Onofré
> > > > > jbonofre@apache.org
> > > > > http://blog.nanthrax.net
> > > > > Talend - http://www.talend.com
> > > > >
> > > >
> > >
> >
>
Re: Spark Runner Technical Vision
Posted by James Malone <ja...@google.com.INVALID>.
Makes complete sense to me. :)
On Wed, Feb 10, 2016 at 10:55 PM, Tyler Akidau <ta...@google.com.invalid>
wrote:
> +1
>
> Although, should/could we have a shared folder to store things in as well
> (since we probably don't want to be linking everything from the website,
> but it'd be nice to have a consistent place for all relevant docs to be
> found)?
>
> -Tyler
>
> On Wed, Feb 10, 2016 at 10:51 PM James Malone
> <ja...@google.com.invalid> wrote:
>
> > Any objections if I link to this (and other design) doc on the (upcoming)
> > project site (beam.incubator.apache.org)?
> >
> > On Wed, Feb 10, 2016 at 10:31 PM, Tyler Akidau
> <takidau@google.com.invalid
> > >
> > wrote:
> >
> > > +1. Added a bunch of comments. Thanks!
> > >
> > > On Wed, Feb 10, 2016 at 9:35 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
> > > wrote:
> > >
> > > > Thanks Amit.
> > > >
> > > > Regards
> > > > JB
> > > >
> > > > On 02/10/2016 07:46 PM, Amit Sela wrote:
> > > > > Following the technical vision Frances sent, and following the
> > previous
> > > > > document I wrote on the spark runner gaps, I'm attaching a
> suggested
> > > > > technical vision to the Spark runner.
> > > > >
> > > > > I tried to be inline with the project's technical vision both in
> > > > timeline,
> > > > > and in document style. I suggest that once the ASF Wiki is up,
> we'll
> > > have
> > > > > roadmap for runners as well.
> > > > >
> > > > > Thanks,
> > > > > Amit
> > > > >
> > > > >
> > > >
> > >
> >
> https://docs.google.com/document/d/1y4qlQinjjrusGWlgq-mYmbxRW2z7-_X5Xax-GG0YsC0/edit?usp=sharing
> > > > >
> > > >
> > > > --
> > > > Jean-Baptiste Onofré
> > > > jbonofre@apache.org
> > > > http://blog.nanthrax.net
> > > > Talend - http://www.talend.com
> > > >
> > >
> >
>
Re: Spark Runner Technical Vision
Posted by Tyler Akidau <ta...@google.com.INVALID>.
+1
Although, should/could we have a shared folder to store things in as well
(since we probably don't want to be linking everything from the website,
but it'd be nice to have a consistent place for all relevant docs to be
found)?
-Tyler
On Wed, Feb 10, 2016 at 10:51 PM James Malone
<ja...@google.com.invalid> wrote:
> Any objections if I link to this (and other design) doc on the (upcoming)
> project site (beam.incubator.apache.org)?
>
> On Wed, Feb 10, 2016 at 10:31 PM, Tyler Akidau <takidau@google.com.invalid
> >
> wrote:
>
> > +1. Added a bunch of comments. Thanks!
> >
> > On Wed, Feb 10, 2016 at 9:35 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
> > wrote:
> >
> > > Thanks Amit.
> > >
> > > Regards
> > > JB
> > >
> > > On 02/10/2016 07:46 PM, Amit Sela wrote:
> > > > Following the technical vision Frances sent, and following the
> previous
> > > > document I wrote on the spark runner gaps, I'm attaching a suggested
> > > > technical vision to the Spark runner.
> > > >
> > > > I tried to be inline with the project's technical vision both in
> > > timeline,
> > > > and in document style. I suggest that once the ASF Wiki is up, we'll
> > have
> > > > roadmap for runners as well.
> > > >
> > > > Thanks,
> > > > Amit
> > > >
> > > >
> > >
> >
> https://docs.google.com/document/d/1y4qlQinjjrusGWlgq-mYmbxRW2z7-_X5Xax-GG0YsC0/edit?usp=sharing
> > > >
> > >
> > > --
> > > Jean-Baptiste Onofré
> > > jbonofre@apache.org
> > > http://blog.nanthrax.net
> > > Talend - http://www.talend.com
> > >
> >
>
Re: Spark Runner Technical Vision
Posted by Amit Sela <am...@gmail.com>.
+1
On Thu, Feb 11, 2016, 08:56 Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
> +1
>
> I also provided some suggestions on Frances' doc that we can link on the
> website.
>
> Regards
> JB
>
> On 02/11/2016 07:50 AM, James Malone wrote:
> > Any objections if I link to this (and other design) doc on the (upcoming)
> > project site (beam.incubator.apache.org)?
> >
> > On Wed, Feb 10, 2016 at 10:31 PM, Tyler Akidau
> <ta...@google.com.invalid>
> > wrote:
> >
> >> +1. Added a bunch of comments. Thanks!
> >>
> >> On Wed, Feb 10, 2016 at 9:35 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
> >> wrote:
> >>
> >>> Thanks Amit.
> >>>
> >>> Regards
> >>> JB
> >>>
> >>> On 02/10/2016 07:46 PM, Amit Sela wrote:
> >>>> Following the technical vision Frances sent, and following the
> previous
> >>>> document I wrote on the spark runner gaps, I'm attaching a suggested
> >>>> technical vision to the Spark runner.
> >>>>
> >>>> I tried to be inline with the project's technical vision both in
> >>> timeline,
> >>>> and in document style. I suggest that once the ASF Wiki is up, we'll
> >> have
> >>>> roadmap for runners as well.
> >>>>
> >>>> Thanks,
> >>>> Amit
> >>>>
> >>>>
> >>>
> >>
> https://docs.google.com/document/d/1y4qlQinjjrusGWlgq-mYmbxRW2z7-_X5Xax-GG0YsC0/edit?usp=sharing
> >>>>
> >>>
> >>> --
> >>> Jean-Baptiste Onofré
> >>> jbonofre@apache.org
> >>> http://blog.nanthrax.net
> >>> Talend - http://www.talend.com
> >>>
> >>
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: Spark Runner Technical Vision
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
+1
I also provided some suggestions on Frances' doc that we can link on the
website.
Regards
JB
On 02/11/2016 07:50 AM, James Malone wrote:
> Any objections if I link to this (and other design) doc on the (upcoming)
> project site (beam.incubator.apache.org)?
>
> On Wed, Feb 10, 2016 at 10:31 PM, Tyler Akidau <ta...@google.com.invalid>
> wrote:
>
>> +1. Added a bunch of comments. Thanks!
>>
>> On Wed, Feb 10, 2016 at 9:35 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>> wrote:
>>
>>> Thanks Amit.
>>>
>>> Regards
>>> JB
>>>
>>> On 02/10/2016 07:46 PM, Amit Sela wrote:
>>>> Following the technical vision Frances sent, and following the previous
>>>> document I wrote on the spark runner gaps, I'm attaching a suggested
>>>> technical vision to the Spark runner.
>>>>
>>>> I tried to be inline with the project's technical vision both in
>>> timeline,
>>>> and in document style. I suggest that once the ASF Wiki is up, we'll
>> have
>>>> roadmap for runners as well.
>>>>
>>>> Thanks,
>>>> Amit
>>>>
>>>>
>>>
>> https://docs.google.com/document/d/1y4qlQinjjrusGWlgq-mYmbxRW2z7-_X5Xax-GG0YsC0/edit?usp=sharing
>>>>
>>>
>>> --
>>> Jean-Baptiste Onofré
>>> jbonofre@apache.org
>>> http://blog.nanthrax.net
>>> Talend - http://www.talend.com
>>>
>>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Spark Runner Technical Vision
Posted by James Malone <ja...@google.com.INVALID>.
Any objections if I link to this (and other design) doc on the (upcoming)
project site (beam.incubator.apache.org)?
On Wed, Feb 10, 2016 at 10:31 PM, Tyler Akidau <ta...@google.com.invalid>
wrote:
> +1. Added a bunch of comments. Thanks!
>
> On Wed, Feb 10, 2016 at 9:35 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
> > Thanks Amit.
> >
> > Regards
> > JB
> >
> > On 02/10/2016 07:46 PM, Amit Sela wrote:
> > > Following the technical vision Frances sent, and following the previous
> > > document I wrote on the spark runner gaps, I'm attaching a suggested
> > > technical vision to the Spark runner.
> > >
> > > I tried to be inline with the project's technical vision both in
> > timeline,
> > > and in document style. I suggest that once the ASF Wiki is up, we'll
> have
> > > roadmap for runners as well.
> > >
> > > Thanks,
> > > Amit
> > >
> > >
> >
> https://docs.google.com/document/d/1y4qlQinjjrusGWlgq-mYmbxRW2z7-_X5Xax-GG0YsC0/edit?usp=sharing
> > >
> >
> > --
> > Jean-Baptiste Onofré
> > jbonofre@apache.org
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
> >
>
Re: Spark Runner Technical Vision
Posted by Tyler Akidau <ta...@google.com.INVALID>.
+1. Added a bunch of comments. Thanks!
On Wed, Feb 10, 2016 at 9:35 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Thanks Amit.
>
> Regards
> JB
>
> On 02/10/2016 07:46 PM, Amit Sela wrote:
> > Following the technical vision Frances sent, and following the previous
> > document I wrote on the spark runner gaps, I'm attaching a suggested
> > technical vision to the Spark runner.
> >
> > I tried to be inline with the project's technical vision both in
> timeline,
> > and in document style. I suggest that once the ASF Wiki is up, we'll have
> > roadmap for runners as well.
> >
> > Thanks,
> > Amit
> >
> >
> https://docs.google.com/document/d/1y4qlQinjjrusGWlgq-mYmbxRW2z7-_X5Xax-GG0YsC0/edit?usp=sharing
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: Spark Runner Technical Vision
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Thanks Amit.
Regards
JB
On 02/10/2016 07:46 PM, Amit Sela wrote:
> Following the technical vision Frances sent, and following the previous
> document I wrote on the spark runner gaps, I'm attaching a suggested
> technical vision to the Spark runner.
>
> I tried to be inline with the project's technical vision both in timeline,
> and in document style. I suggest that once the ASF Wiki is up, we'll have
> roadmap for runners as well.
>
> Thanks,
> Amit
>
> https://docs.google.com/document/d/1y4qlQinjjrusGWlgq-mYmbxRW2z7-_X5Xax-GG0YsC0/edit?usp=sharing
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com