You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Kenneth Knowles <ke...@apache.org> on 2017/05/22 05:12:09 UTC
New doc: Beam Runner Guide
Hi all,
I spent a little time since 2.0.0 to write a guide for runner authors,
aimed at covering everything you need to know and what you need to do to
get a Beam runner off the ground:
https://s.apache.org/beam-runner-guide
It is meant as a complement to Luke's great docs on the Fn API, covering
the other bits that are not the actual UDF invocations. Between the Runner
[API] docs and the Fn API docs, I'm really excited about the Beam
portability story.
Our current runners have already gone through a lot of this, but they did
not all hit the same snags. So I hope this will also be a place where
lessons learned while polishing one runner can be stored and applied to
others.
I think this should be a living document that incorporates lessons from
other runner authors and contributors, so I am shipping it with TODOs,
comments, and some fairly brief sections where I am hoping folks will enjoy
contributing :-). At some point this content could be incorporated
somewhere deep in the web page, not meant for pipeline authors, but let's
wait and see on that.
Kenn
Re: New doc: Beam Runner Guide
Posted by Kenneth Knowles <kl...@google.com.INVALID>.
On Tue, May 23, 2017 at 9:18 AM, Davor Bonaci <da...@apache.org> wrote:
> Great work!
>
> Once comments are resolved in the document, can we move this to the
> Contribute section of the website? I think that would make it much more
> discoverable to everyone.
>
Definitely. I want to make some of the "easy path" implementations real
step-by-step tutorials with pointers to code (since the Javadoc is not for
end-users so not published) and with that and other suggested changes it
should reach a decent quality bar to begin life as a markdown PR.
Kenn
> On Mon, May 22, 2017 at 1:33 AM, Aljoscha Krettek <al...@apache.org>
> wrote:
>
> > Cool, I’ll give this a read once I’m back from conference season. :-)
> >
> > > On 22. May 2017, at 07:12, Kenneth Knowles <ke...@apache.org> wrote:
> > >
> > > Hi all,
> > >
> > > I spent a little time since 2.0.0 to write a guide for runner authors,
> > > aimed at covering everything you need to know and what you need to do
> to
> > > get a Beam runner off the ground:
> > >
> > > https://s.apache.org/beam-runner-guide
> > >
> > > It is meant as a complement to Luke's great docs on the Fn API,
> covering
> > > the other bits that are not the actual UDF invocations. Between the
> > Runner
> > > [API] docs and the Fn API docs, I'm really excited about the Beam
> > > portability story.
> > >
> > > Our current runners have already gone through a lot of this, but they
> did
> > > not all hit the same snags. So I hope this will also be a place where
> > > lessons learned while polishing one runner can be stored and applied to
> > > others.
> > >
> > > I think this should be a living document that incorporates lessons from
> > > other runner authors and contributors, so I am shipping it with TODOs,
> > > comments, and some fairly brief sections where I am hoping folks will
> > enjoy
> > > contributing :-). At some point this content could be incorporated
> > > somewhere deep in the web page, not meant for pipeline authors, but
> let's
> > > wait and see on that.
> > >
> > > Kenn
> >
> >
>
Re: New doc: Beam Runner Guide
Posted by Davor Bonaci <da...@apache.org>.
Great work!
Once comments are resolved in the document, can we move this to the
Contribute section of the website? I think that would make it much more
discoverable to everyone.
On Mon, May 22, 2017 at 1:33 AM, Aljoscha Krettek <al...@apache.org>
wrote:
> Cool, I’ll give this a read once I’m back from conference season. :-)
>
> > On 22. May 2017, at 07:12, Kenneth Knowles <ke...@apache.org> wrote:
> >
> > Hi all,
> >
> > I spent a little time since 2.0.0 to write a guide for runner authors,
> > aimed at covering everything you need to know and what you need to do to
> > get a Beam runner off the ground:
> >
> > https://s.apache.org/beam-runner-guide
> >
> > It is meant as a complement to Luke's great docs on the Fn API, covering
> > the other bits that are not the actual UDF invocations. Between the
> Runner
> > [API] docs and the Fn API docs, I'm really excited about the Beam
> > portability story.
> >
> > Our current runners have already gone through a lot of this, but they did
> > not all hit the same snags. So I hope this will also be a place where
> > lessons learned while polishing one runner can be stored and applied to
> > others.
> >
> > I think this should be a living document that incorporates lessons from
> > other runner authors and contributors, so I am shipping it with TODOs,
> > comments, and some fairly brief sections where I am hoping folks will
> enjoy
> > contributing :-). At some point this content could be incorporated
> > somewhere deep in the web page, not meant for pipeline authors, but let's
> > wait and see on that.
> >
> > Kenn
>
>
Re: New doc: Beam Runner Guide
Posted by Aljoscha Krettek <al...@apache.org>.
Cool, I’ll give this a read once I’m back from conference season. :-)
> On 22. May 2017, at 07:12, Kenneth Knowles <ke...@apache.org> wrote:
>
> Hi all,
>
> I spent a little time since 2.0.0 to write a guide for runner authors,
> aimed at covering everything you need to know and what you need to do to
> get a Beam runner off the ground:
>
> https://s.apache.org/beam-runner-guide
>
> It is meant as a complement to Luke's great docs on the Fn API, covering
> the other bits that are not the actual UDF invocations. Between the Runner
> [API] docs and the Fn API docs, I'm really excited about the Beam
> portability story.
>
> Our current runners have already gone through a lot of this, but they did
> not all hit the same snags. So I hope this will also be a place where
> lessons learned while polishing one runner can be stored and applied to
> others.
>
> I think this should be a living document that incorporates lessons from
> other runner authors and contributors, so I am shipping it with TODOs,
> comments, and some fairly brief sections where I am hoping folks will enjoy
> contributing :-). At some point this content could be incorporated
> somewhere deep in the web page, not meant for pipeline authors, but let's
> wait and see on that.
>
> Kenn