You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Sandeep Deshmukh <sa...@datatorrent.com> on 2016/11/15 15:00:43 UTC
Including Apex runner in Beam tutorial at Strata - Singapore
Dear Beam Community,
There is a Beam tutorial in Strata-Singapore. I would like to explore
possibility of including the Apex runner as a part of that tutorial. As
Apex runner is recently merged into master branch of Beam, it would be of
interest to many people.
Please let us know if we can do so. I can accordingly work on the same.
Regards,
Sandeep
Re: Including Apex runner in Beam tutorial at Strata - Singapore
Posted by Sandeep Deshmukh <sa...@datatorrent.com>.
Thanks JB. I will get started on this.
Regards,
Sandeep
On 15-Nov-2016 8:34 PM, "Jean-Baptiste Onofré" <jb...@nanthrax.net> wrote:
> Hi Sandeep,
>
> Great news !
>
> Yes, you can definitely do a demo using the Apex runner. It's what Dan and
> I are also planning during ApacheCon this week: same Wordcount example
> running on different execution engines.
>
> Maybe this blog could help you to prepare the demo:
> http://blog.nanthrax.net/2016/08/apache-beam-in-action-same-
> code-several-execution-engines/
>
> By the way, I will propose a PR to "merge" those blog to Beam website.
>
> Regards
> JB
>
> On 11/15/2016 04:00 PM, Sandeep Deshmukh wrote:
>
>> Dear Beam Community,
>>
>> There is a Beam tutorial in Strata-Singapore. I would like to explore
>> possibility of including the Apex runner as a part of that tutorial. As
>> Apex runner is recently merged into master branch of Beam, it would be of
>> interest to many people.
>>
>> Please let us know if we can do so. I can accordingly work on the same.
>>
>> Regards,
>> Sandeep
>>
>>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: Including Apex runner in Beam tutorial at Strata - Singapore
Posted by Sandeep Deshmukh <sa...@datatorrent.com>.
Thanks Thomas for the suggestions. I will work on 2nd and 3rd part.
Connecting with Tyler Akidau offline.
Regards,
Sandeep
On Wed, Nov 16, 2016 at 4:40 AM, Thomas Weise <th...@apache.org> wrote:
> The runner currently only executes in embedded mode and I'm not sure that
> will change prior to Strata due to dependency on next Apex core release.
>
> I would suggest to aim for the following:
>
>
> - Create a sample wordcount project that produces the packaging that is
> required to launch the pipeline on YARN (I can take this up).
> - Create the instructions for folks to setup a YARN cluster locally. I
> think a good option for the demo would be to use the DataTorrent
> sandbox.
> - Create tutorial documentation that shows how to alter the example,
> launch it using the Apex CLI and also how to observe the application
> through the UI (comes with the sandbox).
>
>
> Thanks,
> Thomas
>
>
>
>
>
> On Tue, Nov 15, 2016 at 7:07 PM, Tyler Akidau <ta...@google.com.invalid>
> wrote:
>
> > Yes, I'll be giving the tutorial in Singapore w/ Dan Halperin. We'd be
> > happy to include Apex in the demos as part of that. Let's sync up offline
> > about what that will entail.
> >
> > -Tyler
> >
> >
> > On Tue, Nov 15, 2016 at 10:02 AM Davor Bonaci <da...@google.com.invalid>
> > wrote:
> >
> > > Hi Sandeep,
> > > It would be great to include the Apex runner as a part of any tutorial
> > > going forward. I suspect we'll have the 0.4.0-incubating release
> > completed
> > > just before Strata Singapore, which will the first release with the
> Apex
> > > runner, so that aligns quite nicely.
> > >
> > > Are you planning to attend Strata Singapore? If so, I'd encourage you
> to
> > > reach out to Tyler Akidau offline, who's leading the tutorial on this
> > > conference.
> > >
> > > Davor
> > >
> > > On Tue, Nov 15, 2016 at 7:04 AM, Jean-Baptiste Onofré <jb@nanthrax.net
> >
> > > wrote:
> > >
> > > > Hi Sandeep,
> > > >
> > > > Great news !
> > > >
> > > > Yes, you can definitely do a demo using the Apex runner. It's what
> Dan
> > > and
> > > > I are also planning during ApacheCon this week: same Wordcount
> example
> > > > running on different execution engines.
> > > >
> > > > Maybe this blog could help you to prepare the demo:
> > > > http://blog.nanthrax.net/2016/08/apache-beam-in-action-same-
> > > > code-several-execution-engines/
> > > >
> > > > By the way, I will propose a PR to "merge" those blog to Beam
> website.
> > > >
> > > > Regards
> > > > JB
> > > >
> > > >
> > > > On 11/15/2016 04:00 PM, Sandeep Deshmukh wrote:
> > > >
> > > >> Dear Beam Community,
> > > >>
> > > >> There is a Beam tutorial in Strata-Singapore. I would like to
> explore
> > > >> possibility of including the Apex runner as a part of that tutorial.
> > As
> > > >> Apex runner is recently merged into master branch of Beam, it would
> be
> > > of
> > > >> interest to many people.
> > > >>
> > > >> Please let us know if we can do so. I can accordingly work on the
> > same.
> > > >>
> > > >> Regards,
> > > >> Sandeep
> > > >>
> > > >>
> > > > --
> > > > Jean-Baptiste Onofré
> > > > jbonofre@apache.org
> > > > http://blog.nanthrax.net
> > > > Talend - http://www.talend.com
> > > >
> > >
> >
>
Re: Including Apex runner in Beam tutorial at Strata - Singapore
Posted by Thomas Weise <th...@apache.org>.
The runner currently only executes in embedded mode and I'm not sure that
will change prior to Strata due to dependency on next Apex core release.
I would suggest to aim for the following:
- Create a sample wordcount project that produces the packaging that is
required to launch the pipeline on YARN (I can take this up).
- Create the instructions for folks to setup a YARN cluster locally. I
think a good option for the demo would be to use the DataTorrent sandbox.
- Create tutorial documentation that shows how to alter the example,
launch it using the Apex CLI and also how to observe the application
through the UI (comes with the sandbox).
Thanks,
Thomas
On Tue, Nov 15, 2016 at 7:07 PM, Tyler Akidau <ta...@google.com.invalid>
wrote:
> Yes, I'll be giving the tutorial in Singapore w/ Dan Halperin. We'd be
> happy to include Apex in the demos as part of that. Let's sync up offline
> about what that will entail.
>
> -Tyler
>
>
> On Tue, Nov 15, 2016 at 10:02 AM Davor Bonaci <da...@google.com.invalid>
> wrote:
>
> > Hi Sandeep,
> > It would be great to include the Apex runner as a part of any tutorial
> > going forward. I suspect we'll have the 0.4.0-incubating release
> completed
> > just before Strata Singapore, which will the first release with the Apex
> > runner, so that aligns quite nicely.
> >
> > Are you planning to attend Strata Singapore? If so, I'd encourage you to
> > reach out to Tyler Akidau offline, who's leading the tutorial on this
> > conference.
> >
> > Davor
> >
> > On Tue, Nov 15, 2016 at 7:04 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
> > wrote:
> >
> > > Hi Sandeep,
> > >
> > > Great news !
> > >
> > > Yes, you can definitely do a demo using the Apex runner. It's what Dan
> > and
> > > I are also planning during ApacheCon this week: same Wordcount example
> > > running on different execution engines.
> > >
> > > Maybe this blog could help you to prepare the demo:
> > > http://blog.nanthrax.net/2016/08/apache-beam-in-action-same-
> > > code-several-execution-engines/
> > >
> > > By the way, I will propose a PR to "merge" those blog to Beam website.
> > >
> > > Regards
> > > JB
> > >
> > >
> > > On 11/15/2016 04:00 PM, Sandeep Deshmukh wrote:
> > >
> > >> Dear Beam Community,
> > >>
> > >> There is a Beam tutorial in Strata-Singapore. I would like to explore
> > >> possibility of including the Apex runner as a part of that tutorial.
> As
> > >> Apex runner is recently merged into master branch of Beam, it would be
> > of
> > >> interest to many people.
> > >>
> > >> Please let us know if we can do so. I can accordingly work on the
> same.
> > >>
> > >> Regards,
> > >> Sandeep
> > >>
> > >>
> > > --
> > > Jean-Baptiste Onofré
> > > jbonofre@apache.org
> > > http://blog.nanthrax.net
> > > Talend - http://www.talend.com
> > >
> >
>
Re: Including Apex runner in Beam tutorial at Strata - Singapore
Posted by Tyler Akidau <ta...@google.com.INVALID>.
Yes, I'll be giving the tutorial in Singapore w/ Dan Halperin. We'd be
happy to include Apex in the demos as part of that. Let's sync up offline
about what that will entail.
-Tyler
On Tue, Nov 15, 2016 at 10:02 AM Davor Bonaci <da...@google.com.invalid>
wrote:
> Hi Sandeep,
> It would be great to include the Apex runner as a part of any tutorial
> going forward. I suspect we'll have the 0.4.0-incubating release completed
> just before Strata Singapore, which will the first release with the Apex
> runner, so that aligns quite nicely.
>
> Are you planning to attend Strata Singapore? If so, I'd encourage you to
> reach out to Tyler Akidau offline, who's leading the tutorial on this
> conference.
>
> Davor
>
> On Tue, Nov 15, 2016 at 7:04 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
> > Hi Sandeep,
> >
> > Great news !
> >
> > Yes, you can definitely do a demo using the Apex runner. It's what Dan
> and
> > I are also planning during ApacheCon this week: same Wordcount example
> > running on different execution engines.
> >
> > Maybe this blog could help you to prepare the demo:
> > http://blog.nanthrax.net/2016/08/apache-beam-in-action-same-
> > code-several-execution-engines/
> >
> > By the way, I will propose a PR to "merge" those blog to Beam website.
> >
> > Regards
> > JB
> >
> >
> > On 11/15/2016 04:00 PM, Sandeep Deshmukh wrote:
> >
> >> Dear Beam Community,
> >>
> >> There is a Beam tutorial in Strata-Singapore. I would like to explore
> >> possibility of including the Apex runner as a part of that tutorial. As
> >> Apex runner is recently merged into master branch of Beam, it would be
> of
> >> interest to many people.
> >>
> >> Please let us know if we can do so. I can accordingly work on the same.
> >>
> >> Regards,
> >> Sandeep
> >>
> >>
> > --
> > Jean-Baptiste Onofré
> > jbonofre@apache.org
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
> >
>
Re: Including Apex runner in Beam tutorial at Strata - Singapore
Posted by Davor Bonaci <da...@google.com.INVALID>.
Hi Sandeep,
It would be great to include the Apex runner as a part of any tutorial
going forward. I suspect we'll have the 0.4.0-incubating release completed
just before Strata Singapore, which will the first release with the Apex
runner, so that aligns quite nicely.
Are you planning to attend Strata Singapore? If so, I'd encourage you to
reach out to Tyler Akidau offline, who's leading the tutorial on this
conference.
Davor
On Tue, Nov 15, 2016 at 7:04 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Hi Sandeep,
>
> Great news !
>
> Yes, you can definitely do a demo using the Apex runner. It's what Dan and
> I are also planning during ApacheCon this week: same Wordcount example
> running on different execution engines.
>
> Maybe this blog could help you to prepare the demo:
> http://blog.nanthrax.net/2016/08/apache-beam-in-action-same-
> code-several-execution-engines/
>
> By the way, I will propose a PR to "merge" those blog to Beam website.
>
> Regards
> JB
>
>
> On 11/15/2016 04:00 PM, Sandeep Deshmukh wrote:
>
>> Dear Beam Community,
>>
>> There is a Beam tutorial in Strata-Singapore. I would like to explore
>> possibility of including the Apex runner as a part of that tutorial. As
>> Apex runner is recently merged into master branch of Beam, it would be of
>> interest to many people.
>>
>> Please let us know if we can do so. I can accordingly work on the same.
>>
>> Regards,
>> Sandeep
>>
>>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: Including Apex runner in Beam tutorial at Strata - Singapore
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Sandeep,
Great news !
Yes, you can definitely do a demo using the Apex runner. It's what Dan
and I are also planning during ApacheCon this week: same Wordcount
example running on different execution engines.
Maybe this blog could help you to prepare the demo:
http://blog.nanthrax.net/2016/08/apache-beam-in-action-same-code-several-execution-engines/
By the way, I will propose a PR to "merge" those blog to Beam website.
Regards
JB
On 11/15/2016 04:00 PM, Sandeep Deshmukh wrote:
> Dear Beam Community,
>
> There is a Beam tutorial in Strata-Singapore. I would like to explore
> possibility of including the Apex runner as a part of that tutorial. As
> Apex runner is recently merged into master branch of Beam, it would be of
> interest to many people.
>
> Please let us know if we can do so. I can accordingly work on the same.
>
> Regards,
> Sandeep
>
--
Jean-Baptiste Onofr�
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com