You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Henry Suryawirawan <hs...@google.com> on 2019/04/18 03:30:41 UTC
Contributing Beam Kata (Java & Python)
Hi Beam Devs,
I have been working on what I call Beam Kata, a set of interactive Beam
coding exercises (i.e. code kata) that aids in learning Beam programming
hands-on. It is built based on JetBrains Educational Products (
https://www.jetbrains.com/education/).
The objective of Beam Kata is to provide a structured hands-on learning
experience for people to learn about Beam and its SDK by solving focused
mini problems with gradually increasing complexity.
I have used the Kata to conduct Beam workshops, teaching the participants
on how to code in Beam, and so far the feedback has been great.
They particularly like the hands-on and how it is closely integrated with
the IDE, giving the best developer experience, especially for
IntelliJ/PyCharm developers.
I would like to contribute this to the Beam repository and would like to
check with you whether this is something that would be useful as an
addition.
If so, can you advise me in which directory I should commit my work?
If you need further information, please do let me know.
As of the moment, I have created lessons for both Java & Python SDK.
The current lessons are:
# Introduction
* Hello Beam
# Core Transforms
* Map
** ParDo
** ParDo OneToMany
** MapElements / Map
** FlatMapElements / FlatMap
* GroupByKey
* CoGroupByKey
* Combine
** Simple Function
** CombineFn
** BinaryCombineFn
** Combine PerKey
* Flatten
* Partition
# Common Transforms
* Filter
** ParDo
** Filter
* Aggregation
** Count
** Sum
** Avg
** Min / Smallest
** Max / Largest
I have plans to add more lessons to make it more complete, covering more
Beam fundamentals, e.g. composite transform, branching (multiple
transforms, side output), merging (side input), I/O, windowing, streaming,
examples (word count, etc), and possibly use case patterns.
Please find some screenshots on how it looks like for a student:
[image: image.png]
[image: image.png]
Regards,
Henry Suryawirawan
Re: Contributing Beam Kata (Java & Python)
Posted by .
Thanks Kenneth.
Yeah from a glance this might fit with the incubating training project.
As of this moment, the kata is using solely the direct runner as the focus currently is to teach about Beam primitives.
There are ideas on how to also have it running on other runner (e.g. Dataflow), but so far I haven't created it since it will need integration with a GCP project which requires some setup for a student.
On 2019/04/18 21:49:50, Kenneth Knowles <ke...@apache.org> wrote:
> Great! By the way it might fit in with the newly incubating training
> project [1]. Do you go into specifics on using different runners at all?
>
> Kenn
>
> [1] https://wiki.apache.org/incubator/TrainingProposal
>
Re: Contributing Beam Kata (Java & Python)
Posted by Henry Suryawirawan <hs...@google.com>.
Sure, let me try to write one.
I've created a JIRA for it: https://issues.apache.org/jira/browse/BEAM-7332
*From: *Pablo Estrada <pa...@google.com>
*Date: *Thu, May 16, 2019 at 2:20 AM
*To: *dev
I think a blog post is a great idea. Would you be able to write one?
> See here for an example of a PR adding one:
> https://github.com/apache/beam/pull/8391
>
> Best
> -P.
>
> *From: *hsuryawirawan@google.com <hs...@google.com>
> *Date: *Wed, May 15, 2019, 12:51 AM
> *To: * <de...@beam.apache.org>
>
> Thanks for merging it Reuven!
>>
>> Quick question, would it be useful if we write a blog post on the Kata so
>> that we can build more awareness for people to try out?
>> I've also uploaded the course to Stepik which has seamless integration
>> within the IDE for people to easily start the course.
>>
>> On 2019/05/14 20:35:07, Reuven Lax <re...@google.com> wrote:
>> > Merged
>> >
>> > *From: *Reza Rokni <re...@google.com>
>> > *Date: *Tue, May 14, 2019 at 1:29 PM
>> > *To: * <de...@beam.apache.org>
>> > *Cc: *Lars Francke
>> >
>> > +1 :-)
>> > >
>> > > *From: *Lukasz Cwik <lc...@google.com>
>> > > *Date: *Wed, 15 May 2019 at 04:29
>> > > *To: *dev
>> > > *Cc: *Lars Francke
>> > >
>> > > +1
>> > >>
>> > >> *From: *Pablo Estrada <pa...@google.com>
>> > >> *Date: *Tue, May 14, 2019 at 1:27 PM
>> > >> *To: *dev
>> > >> *Cc: *Lars Francke
>> > >>
>> > >> +1 on merging.
>> > >>>
>> > >>> *From: *Reuven Lax <re...@google.com>
>> > >>> *Date: *Tue, May 14, 2019 at 1:23 PM
>> > >>> *To: *dev
>> > >>> *Cc: *Lars Francke
>> > >>>
>> > >>> I've been playing around with this that past day .or two, and it's
>> > >>>> great! I'm inclined to merge this PR (if nobody objects) so that
>> others in
>> > >>>> the community can contribute more training katas.
>> > >>>>
>> > >>>> Reuven
>> > >>>>
>> > >>>> *From: *Ismaël Mejía <ie...@gmail.com>
>> > >>>> *Date: *Tue, Apr 23, 2019 at 6:43 AM
>> > >>>> *To: *Lars Francke
>> > >>>> *Cc: * <de...@beam.apache.org>
>> > >>>>
>> > >>>> Thanks for answering Lars,
>> > >>>>>
>> > >>>>> The 'interesting' part is that the tutorial has a full IDE
>> integrated
>> > >>>>> experience based on the Jetbrains edu platform [1]. So maybe
>> > >>>>> interesting to see if it could make sense to have projects like
>> this
>> > >>>>> in the new trainings incubator project or if they became too
>> platform
>> > >>>>> constrained.
>> > >>>>>
>> > >>>>> This contribution is valuable for Beam but the community may
>> decide
>> > >>>>> that it makes sense for it to live at some moment at the trainings
>> > >>>>> project. I suppose also Henry could be interested in taking a
>> look at
>> > >>>>> this [2].
>> > >>>>>
>> > >>>>> [1] https://www.jetbrains.com/education/
>> > >>>>> [2] https://incubator.apache.org/clutch/training.html
>> > >>>>>
>> > >>>>> On Tue, Apr 23, 2019 at 3:00 PM Lars Francke <
>> lars.francke@gmail.com>
>> > >>>>> wrote:
>> > >>>>> >
>> > >>>>> > Thanks Ismaël.
>> > >>>>> >
>> > >>>>> > I must admit I'm a tad confused. What has JetBrains got to do
>> with
>> > >>>>> this?
>> > >>>>> > This looks pretty cool and specific to Beam though, or is this
>> more
>> > >>>>> generic?
>> > >>>>> > But yeah something along those lines could be interesting for
>> > >>>>> hands-on type things in training.
>> > >>>>> >
>> > >>>>> > On Fri, Apr 19, 2019 at 12:10 PM Ismaël Mejía <
>> iemejia@gmail.com>
>> > >>>>> wrote:
>> > >>>>> >>
>> > >>>>> >> +lars.francke@gmail.com who is in the Apache training project
>> and
>> > >>>>> may
>> > >>>>> >> be interested in this one or at least the JetBrains like
>> approach.
>> > >>>>> >>
>> > >>>>> >> On Fri, Apr 19, 2019 at 12:01 PM Ismaël Mejía <
>> iemejia@gmail.com>
>> > >>>>> wrote:
>> > >>>>> >> >
>> > >>>>> >> > This looks great, nice for bringing this to the project
>> Henry!
>> > >>>>> >> >
>> > >>>>> >> > On Fri, Apr 19, 2019 at 10:53 AM hsuryawirawan@google.com
>> > >>>>> >> > <hs...@google.com> wrote:
>> > >>>>> >> > >
>> > >>>>> >> > > Thanks Altay.
>> > >>>>> >> > > I'll create it under "learning/" first as this is not
>> exactly
>> > >>>>> example.
>> > >>>>> >> > > Please do let me know if it's not the right place.
>> > >>>>> >> > >
>> > >>>>> >> > > On 2019/04/18 22:49:47, Ahmet Altay <al...@google.com>
>> wrote:
>> > >>>>> >> > > > This looks great.
>> > >>>>> >> > > >
>> > >>>>> >> > > > +David Cavazos <dc...@google.com> was working on
>> > >>>>> interactive colab based
>> > >>>>> >> > > > examples (https://github.com/apache/beam/pull/7679)
>> perhaps
>> > >>>>> we can have a
>> > >>>>> >> > > > shared place for these two similar things.
>> > >>>>> >> > > >
>> > >>>>> >> > >
>> > >>>>>
>> > >>>>
>> > >
>> > > --
>> > >
>> > > This email may be confidential and privileged. If you received this
>> > > communication by mistake, please don't forward it to anyone else,
>> please
>> > > erase all copies and attachments, and please let me know that it has
>> gone
>> > > to the wrong person.
>> > >
>> > > The above terms reflect a potential business arrangement, are provided
>> > > solely as a basis for further discussion, and are not intended to be
>> and do
>> > > not constitute a legally binding obligation. No legally binding
>> obligations
>> > > will be created, implied, or inferred until an agreement in final
>> form is
>> > > executed in writing by all parties involved.
>> > >
>> >
>>
>
Re: Contributing Beam Kata (Java & Python)
Posted by Pablo Estrada <pa...@google.com>.
I think a blog post is a great idea. Would you be able to write one?
See here for an example of a PR adding one:
https://github.com/apache/beam/pull/8391
Best
-P.
*From: *hsuryawirawan@google.com <hs...@google.com>
*Date: *Wed, May 15, 2019, 12:51 AM
*To: * <de...@beam.apache.org>
Thanks for merging it Reuven!
>
> Quick question, would it be useful if we write a blog post on the Kata so
> that we can build more awareness for people to try out?
> I've also uploaded the course to Stepik which has seamless integration
> within the IDE for people to easily start the course.
>
> On 2019/05/14 20:35:07, Reuven Lax <re...@google.com> wrote:
> > Merged
> >
> > *From: *Reza Rokni <re...@google.com>
> > *Date: *Tue, May 14, 2019 at 1:29 PM
> > *To: * <de...@beam.apache.org>
> > *Cc: *Lars Francke
> >
> > +1 :-)
> > >
> > > *From: *Lukasz Cwik <lc...@google.com>
> > > *Date: *Wed, 15 May 2019 at 04:29
> > > *To: *dev
> > > *Cc: *Lars Francke
> > >
> > > +1
> > >>
> > >> *From: *Pablo Estrada <pa...@google.com>
> > >> *Date: *Tue, May 14, 2019 at 1:27 PM
> > >> *To: *dev
> > >> *Cc: *Lars Francke
> > >>
> > >> +1 on merging.
> > >>>
> > >>> *From: *Reuven Lax <re...@google.com>
> > >>> *Date: *Tue, May 14, 2019 at 1:23 PM
> > >>> *To: *dev
> > >>> *Cc: *Lars Francke
> > >>>
> > >>> I've been playing around with this that past day .or two, and it's
> > >>>> great! I'm inclined to merge this PR (if nobody objects) so that
> others in
> > >>>> the community can contribute more training katas.
> > >>>>
> > >>>> Reuven
> > >>>>
> > >>>> *From: *Ismaël Mejía <ie...@gmail.com>
> > >>>> *Date: *Tue, Apr 23, 2019 at 6:43 AM
> > >>>> *To: *Lars Francke
> > >>>> *Cc: * <de...@beam.apache.org>
> > >>>>
> > >>>> Thanks for answering Lars,
> > >>>>>
> > >>>>> The 'interesting' part is that the tutorial has a full IDE
> integrated
> > >>>>> experience based on the Jetbrains edu platform [1]. So maybe
> > >>>>> interesting to see if it could make sense to have projects like
> this
> > >>>>> in the new trainings incubator project or if they became too
> platform
> > >>>>> constrained.
> > >>>>>
> > >>>>> This contribution is valuable for Beam but the community may decide
> > >>>>> that it makes sense for it to live at some moment at the trainings
> > >>>>> project. I suppose also Henry could be interested in taking a look
> at
> > >>>>> this [2].
> > >>>>>
> > >>>>> [1] https://www.jetbrains.com/education/
> > >>>>> [2] https://incubator.apache.org/clutch/training.html
> > >>>>>
> > >>>>> On Tue, Apr 23, 2019 at 3:00 PM Lars Francke <
> lars.francke@gmail.com>
> > >>>>> wrote:
> > >>>>> >
> > >>>>> > Thanks Ismaël.
> > >>>>> >
> > >>>>> > I must admit I'm a tad confused. What has JetBrains got to do
> with
> > >>>>> this?
> > >>>>> > This looks pretty cool and specific to Beam though, or is this
> more
> > >>>>> generic?
> > >>>>> > But yeah something along those lines could be interesting for
> > >>>>> hands-on type things in training.
> > >>>>> >
> > >>>>> > On Fri, Apr 19, 2019 at 12:10 PM Ismaël Mejía <iemejia@gmail.com
> >
> > >>>>> wrote:
> > >>>>> >>
> > >>>>> >> +lars.francke@gmail.com who is in the Apache training project
> and
> > >>>>> may
> > >>>>> >> be interested in this one or at least the JetBrains like
> approach.
> > >>>>> >>
> > >>>>> >> On Fri, Apr 19, 2019 at 12:01 PM Ismaël Mejía <
> iemejia@gmail.com>
> > >>>>> wrote:
> > >>>>> >> >
> > >>>>> >> > This looks great, nice for bringing this to the project Henry!
> > >>>>> >> >
> > >>>>> >> > On Fri, Apr 19, 2019 at 10:53 AM hsuryawirawan@google.com
> > >>>>> >> > <hs...@google.com> wrote:
> > >>>>> >> > >
> > >>>>> >> > > Thanks Altay.
> > >>>>> >> > > I'll create it under "learning/" first as this is not
> exactly
> > >>>>> example.
> > >>>>> >> > > Please do let me know if it's not the right place.
> > >>>>> >> > >
> > >>>>> >> > > On 2019/04/18 22:49:47, Ahmet Altay <al...@google.com>
> wrote:
> > >>>>> >> > > > This looks great.
> > >>>>> >> > > >
> > >>>>> >> > > > +David Cavazos <dc...@google.com> was working on
> > >>>>> interactive colab based
> > >>>>> >> > > > examples (https://github.com/apache/beam/pull/7679)
> perhaps
> > >>>>> we can have a
> > >>>>> >> > > > shared place for these two similar things.
> > >>>>> >> > > >
> > >>>>> >> > >
> > >>>>>
> > >>>>
> > >
> > > --
> > >
> > > This email may be confidential and privileged. If you received this
> > > communication by mistake, please don't forward it to anyone else,
> please
> > > erase all copies and attachments, and please let me know that it has
> gone
> > > to the wrong person.
> > >
> > > The above terms reflect a potential business arrangement, are provided
> > > solely as a basis for further discussion, and are not intended to be
> and do
> > > not constitute a legally binding obligation. No legally binding
> obligations
> > > will be created, implied, or inferred until an agreement in final form
> is
> > > executed in writing by all parties involved.
> > >
> >
>
Re: Contributing Beam Kata (Java & Python)
Posted by .
Btw the Beam Kata courses have been reviewed and endorsed by JetBrains on Stepik.
They are now featured courses, which adds more credibility when the students browse the available courses.
JetBrains is also interested to come up with promotion activities to promote the courses.
Will inform here when I get more information from them.
On 2019/05/16 03:55:30, Henry Suryawirawan <hs...@google.com> wrote:
> Hi Austin,
>
> Yes I agree with you that we need to come up with the publishing process.
> We should publish to Stepik only if it has been merged to master.
> I can add more authors & instructors to the courses on Stepik so that more
> people can update the courses.
> I uploaded the initial version to Stepik in order to get the course ID
> (which is also committed to the repo) and also to easily show it to other
> people who would like to give it a try.
> Last time when I ran training using it, there were setup issues and people
> had difficulty following the instruction.
>
> Another thing that we should think about is how to integrate the Kata
> projects into the CI build process.
> For the Java Kata, it is based on Gradle, while for the Python one, I'm
> currently checking with the JetBrains team if there is a good way of
> executing the tests cause the current way of how the test is written is
> non-standard.
> Can anyone advise on how best the Kata projects should be integrated to the
> CI?
>
>
> *From: *Austin Bennett <wh...@gmail.com>
> *Date: *Wed, May 15, 2019 at 11:49 PM
> *To: *dev
>
> Stepik: should we thinking about release versions for what gets uploaded
> > there? If the point of merging was to encourage additional contributions -
> > then should we also have a mechanism for publishing the updates there? In
> > that case, then uploading to stepik should be part of publishing releases?
> >
> > On Tue, May 14, 2019 at 10:51 PM hsuryawirawan@google.com <
> > hsuryawirawan@google.com> wrote:
> >
> >> Thanks for merging it Reuven!
> >>
> >> Quick question, would it be useful if we write a blog post on the Kata so
> >> that we can build more awareness for people to try out?
> >> I've also uploaded the course to Stepik which has seamless integration
> >> within the IDE for people to easily start the course.
> >>
> >> On 2019/05/14 20:35:07, Reuven Lax <re...@google.com> wrote:
> >> > Merged
> >> >
> >> > *From: *Reza Rokni <re...@google.com>
> >> > *Date: *Tue, May 14, 2019 at 1:29 PM
> >> > *To: * <de...@beam.apache.org>
> >> > *Cc: *Lars Francke
> >> >
> >> > +1 :-)
> >> > >
> >> > > *From: *Lukasz Cwik <lc...@google.com>
> >> > > *Date: *Wed, 15 May 2019 at 04:29
> >> > > *To: *dev
> >> > > *Cc: *Lars Francke
> >> > >
> >> > > +1
> >> > >>
> >> > >> *From: *Pablo Estrada <pa...@google.com>
> >> > >> *Date: *Tue, May 14, 2019 at 1:27 PM
> >> > >> *To: *dev
> >> > >> *Cc: *Lars Francke
> >> > >>
> >> > >> +1 on merging.
> >> > >>>
> >> > >>> *From: *Reuven Lax <re...@google.com>
> >> > >>> *Date: *Tue, May 14, 2019 at 1:23 PM
> >> > >>> *To: *dev
> >> > >>> *Cc: *Lars Francke
> >> > >>>
> >> > >>> I've been playing around with this that past day .or two, and it's
> >> > >>>> great! I'm inclined to merge this PR (if nobody objects) so that
> >> others in
> >> > >>>> the community can contribute more training katas.
> >> > >>>>
> >> > >>>> Reuven
> >> > >>>>
> >> > >>>> *From: *Ismaël Mejía <ie...@gmail.com>
> >> > >>>> *Date: *Tue, Apr 23, 2019 at 6:43 AM
> >> > >>>> *To: *Lars Francke
> >> > >>>> *Cc: * <de...@beam.apache.org>
> >> > >>>>
> >> > >>>> Thanks for answering Lars,
> >> > >>>>>
> >> > >>>>> The 'interesting' part is that the tutorial has a full IDE
> >> integrated
> >> > >>>>> experience based on the Jetbrains edu platform [1]. So maybe
> >> > >>>>> interesting to see if it could make sense to have projects like
> >> this
> >> > >>>>> in the new trainings incubator project or if they became too
> >> platform
> >> > >>>>> constrained.
> >> > >>>>>
> >> > >>>>> This contribution is valuable for Beam but the community may
> >> decide
> >> > >>>>> that it makes sense for it to live at some moment at the trainings
> >> > >>>>> project. I suppose also Henry could be interested in taking a
> >> look at
> >> > >>>>> this [2].
> >> > >>>>>
> >> > >>>>> [1] https://www.jetbrains.com/education/
> >> > >>>>> [2] https://incubator.apache.org/clutch/training.html
> >> > >>>>>
> >> > >>>>> On Tue, Apr 23, 2019 at 3:00 PM Lars Francke <
> >> lars.francke@gmail.com>
> >> > >>>>> wrote:
> >> > >>>>> >
> >> > >>>>> > Thanks Ismaël.
> >> > >>>>> >
> >> > >>>>> > I must admit I'm a tad confused. What has JetBrains got to do
> >> with
> >> > >>>>> this?
> >> > >>>>> > This looks pretty cool and specific to Beam though, or is this
> >> more
> >> > >>>>> generic?
> >> > >>>>> > But yeah something along those lines could be interesting for
> >> > >>>>> hands-on type things in training.
> >> > >>>>> >
> >> > >>>>> > On Fri, Apr 19, 2019 at 12:10 PM Ismaël Mejía <
> >> iemejia@gmail.com>
> >> > >>>>> wrote:
> >> > >>>>> >>
> >> > >>>>> >> +lars.francke@gmail.com who is in the Apache training project
> >> and
> >> > >>>>> may
> >> > >>>>> >> be interested in this one or at least the JetBrains like
> >> approach.
> >> > >>>>> >>
> >> > >>>>> >> On Fri, Apr 19, 2019 at 12:01 PM Ismaël Mejía <
> >> iemejia@gmail.com>
> >> > >>>>> wrote:
> >> > >>>>> >> >
> >> > >>>>> >> > This looks great, nice for bringing this to the project
> >> Henry!
> >> > >>>>> >> >
> >> > >>>>> >> > On Fri, Apr 19, 2019 at 10:53 AM hsuryawirawan@google.com
> >> > >>>>> >> > <hs...@google.com> wrote:
> >> > >>>>> >> > >
> >> > >>>>> >> > > Thanks Altay.
> >> > >>>>> >> > > I'll create it under "learning/" first as this is not
> >> exactly
> >> > >>>>> example.
> >> > >>>>> >> > > Please do let me know if it's not the right place.
> >> > >>>>> >> > >
> >> > >>>>> >> > > On 2019/04/18 22:49:47, Ahmet Altay <al...@google.com>
> >> wrote:
> >> > >>>>> >> > > > This looks great.
> >> > >>>>> >> > > >
> >> > >>>>> >> > > > +David Cavazos <dc...@google.com> was working on
> >> > >>>>> interactive colab based
> >> > >>>>> >> > > > examples (https://github.com/apache/beam/pull/7679)
> >> perhaps
> >> > >>>>> we can have a
> >> > >>>>> >> > > > shared place for these two similar things.
> >> > >>>>> >> > > >
> >> > >>>>> >> > >
> >> > >>>>>
> >> > >>>>
> >> > >
> >> > > --
> >> > >
> >> > > This email may be confidential and privileged. If you received this
> >> > > communication by mistake, please don't forward it to anyone else,
> >> please
> >> > > erase all copies and attachments, and please let me know that it has
> >> gone
> >> > > to the wrong person.
> >> > >
> >> > > The above terms reflect a potential business arrangement, are provided
> >> > > solely as a basis for further discussion, and are not intended to be
> >> and do
> >> > > not constitute a legally binding obligation. No legally binding
> >> obligations
> >> > > will be created, implied, or inferred until an agreement in final
> >> form is
> >> > > executed in writing by all parties involved.
> >> > >
> >> >
> >>
> >
>
Re: Contributing Beam Kata (Java & Python)
Posted by Henry Suryawirawan <hs...@google.com>.
Hi Austin,
Yes I agree with you that we need to come up with the publishing process.
We should publish to Stepik only if it has been merged to master.
I can add more authors & instructors to the courses on Stepik so that more
people can update the courses.
I uploaded the initial version to Stepik in order to get the course ID
(which is also committed to the repo) and also to easily show it to other
people who would like to give it a try.
Last time when I ran training using it, there were setup issues and people
had difficulty following the instruction.
Another thing that we should think about is how to integrate the Kata
projects into the CI build process.
For the Java Kata, it is based on Gradle, while for the Python one, I'm
currently checking with the JetBrains team if there is a good way of
executing the tests cause the current way of how the test is written is
non-standard.
Can anyone advise on how best the Kata projects should be integrated to the
CI?
*From: *Austin Bennett <wh...@gmail.com>
*Date: *Wed, May 15, 2019 at 11:49 PM
*To: *dev
Stepik: should we thinking about release versions for what gets uploaded
> there? If the point of merging was to encourage additional contributions -
> then should we also have a mechanism for publishing the updates there? In
> that case, then uploading to stepik should be part of publishing releases?
>
> On Tue, May 14, 2019 at 10:51 PM hsuryawirawan@google.com <
> hsuryawirawan@google.com> wrote:
>
>> Thanks for merging it Reuven!
>>
>> Quick question, would it be useful if we write a blog post on the Kata so
>> that we can build more awareness for people to try out?
>> I've also uploaded the course to Stepik which has seamless integration
>> within the IDE for people to easily start the course.
>>
>> On 2019/05/14 20:35:07, Reuven Lax <re...@google.com> wrote:
>> > Merged
>> >
>> > *From: *Reza Rokni <re...@google.com>
>> > *Date: *Tue, May 14, 2019 at 1:29 PM
>> > *To: * <de...@beam.apache.org>
>> > *Cc: *Lars Francke
>> >
>> > +1 :-)
>> > >
>> > > *From: *Lukasz Cwik <lc...@google.com>
>> > > *Date: *Wed, 15 May 2019 at 04:29
>> > > *To: *dev
>> > > *Cc: *Lars Francke
>> > >
>> > > +1
>> > >>
>> > >> *From: *Pablo Estrada <pa...@google.com>
>> > >> *Date: *Tue, May 14, 2019 at 1:27 PM
>> > >> *To: *dev
>> > >> *Cc: *Lars Francke
>> > >>
>> > >> +1 on merging.
>> > >>>
>> > >>> *From: *Reuven Lax <re...@google.com>
>> > >>> *Date: *Tue, May 14, 2019 at 1:23 PM
>> > >>> *To: *dev
>> > >>> *Cc: *Lars Francke
>> > >>>
>> > >>> I've been playing around with this that past day .or two, and it's
>> > >>>> great! I'm inclined to merge this PR (if nobody objects) so that
>> others in
>> > >>>> the community can contribute more training katas.
>> > >>>>
>> > >>>> Reuven
>> > >>>>
>> > >>>> *From: *Ismaël Mejía <ie...@gmail.com>
>> > >>>> *Date: *Tue, Apr 23, 2019 at 6:43 AM
>> > >>>> *To: *Lars Francke
>> > >>>> *Cc: * <de...@beam.apache.org>
>> > >>>>
>> > >>>> Thanks for answering Lars,
>> > >>>>>
>> > >>>>> The 'interesting' part is that the tutorial has a full IDE
>> integrated
>> > >>>>> experience based on the Jetbrains edu platform [1]. So maybe
>> > >>>>> interesting to see if it could make sense to have projects like
>> this
>> > >>>>> in the new trainings incubator project or if they became too
>> platform
>> > >>>>> constrained.
>> > >>>>>
>> > >>>>> This contribution is valuable for Beam but the community may
>> decide
>> > >>>>> that it makes sense for it to live at some moment at the trainings
>> > >>>>> project. I suppose also Henry could be interested in taking a
>> look at
>> > >>>>> this [2].
>> > >>>>>
>> > >>>>> [1] https://www.jetbrains.com/education/
>> > >>>>> [2] https://incubator.apache.org/clutch/training.html
>> > >>>>>
>> > >>>>> On Tue, Apr 23, 2019 at 3:00 PM Lars Francke <
>> lars.francke@gmail.com>
>> > >>>>> wrote:
>> > >>>>> >
>> > >>>>> > Thanks Ismaël.
>> > >>>>> >
>> > >>>>> > I must admit I'm a tad confused. What has JetBrains got to do
>> with
>> > >>>>> this?
>> > >>>>> > This looks pretty cool and specific to Beam though, or is this
>> more
>> > >>>>> generic?
>> > >>>>> > But yeah something along those lines could be interesting for
>> > >>>>> hands-on type things in training.
>> > >>>>> >
>> > >>>>> > On Fri, Apr 19, 2019 at 12:10 PM Ismaël Mejía <
>> iemejia@gmail.com>
>> > >>>>> wrote:
>> > >>>>> >>
>> > >>>>> >> +lars.francke@gmail.com who is in the Apache training project
>> and
>> > >>>>> may
>> > >>>>> >> be interested in this one or at least the JetBrains like
>> approach.
>> > >>>>> >>
>> > >>>>> >> On Fri, Apr 19, 2019 at 12:01 PM Ismaël Mejía <
>> iemejia@gmail.com>
>> > >>>>> wrote:
>> > >>>>> >> >
>> > >>>>> >> > This looks great, nice for bringing this to the project
>> Henry!
>> > >>>>> >> >
>> > >>>>> >> > On Fri, Apr 19, 2019 at 10:53 AM hsuryawirawan@google.com
>> > >>>>> >> > <hs...@google.com> wrote:
>> > >>>>> >> > >
>> > >>>>> >> > > Thanks Altay.
>> > >>>>> >> > > I'll create it under "learning/" first as this is not
>> exactly
>> > >>>>> example.
>> > >>>>> >> > > Please do let me know if it's not the right place.
>> > >>>>> >> > >
>> > >>>>> >> > > On 2019/04/18 22:49:47, Ahmet Altay <al...@google.com>
>> wrote:
>> > >>>>> >> > > > This looks great.
>> > >>>>> >> > > >
>> > >>>>> >> > > > +David Cavazos <dc...@google.com> was working on
>> > >>>>> interactive colab based
>> > >>>>> >> > > > examples (https://github.com/apache/beam/pull/7679)
>> perhaps
>> > >>>>> we can have a
>> > >>>>> >> > > > shared place for these two similar things.
>> > >>>>> >> > > >
>> > >>>>> >> > >
>> > >>>>>
>> > >>>>
>> > >
>> > > --
>> > >
>> > > This email may be confidential and privileged. If you received this
>> > > communication by mistake, please don't forward it to anyone else,
>> please
>> > > erase all copies and attachments, and please let me know that it has
>> gone
>> > > to the wrong person.
>> > >
>> > > The above terms reflect a potential business arrangement, are provided
>> > > solely as a basis for further discussion, and are not intended to be
>> and do
>> > > not constitute a legally binding obligation. No legally binding
>> obligations
>> > > will be created, implied, or inferred until an agreement in final
>> form is
>> > > executed in writing by all parties involved.
>> > >
>> >
>>
>
Re: Contributing Beam Kata (Java & Python)
Posted by Austin Bennett <wh...@gmail.com>.
Stepik: should we thinking about release versions for what gets uploaded
there? If the point of merging was to encourage additional contributions -
then should we also have a mechanism for publishing the updates there? In
that case, then uploading to stepik should be part of publishing releases?
On Tue, May 14, 2019 at 10:51 PM hsuryawirawan@google.com <
hsuryawirawan@google.com> wrote:
> Thanks for merging it Reuven!
>
> Quick question, would it be useful if we write a blog post on the Kata so
> that we can build more awareness for people to try out?
> I've also uploaded the course to Stepik which has seamless integration
> within the IDE for people to easily start the course.
>
> On 2019/05/14 20:35:07, Reuven Lax <re...@google.com> wrote:
> > Merged
> >
> > *From: *Reza Rokni <re...@google.com>
> > *Date: *Tue, May 14, 2019 at 1:29 PM
> > *To: * <de...@beam.apache.org>
> > *Cc: *Lars Francke
> >
> > +1 :-)
> > >
> > > *From: *Lukasz Cwik <lc...@google.com>
> > > *Date: *Wed, 15 May 2019 at 04:29
> > > *To: *dev
> > > *Cc: *Lars Francke
> > >
> > > +1
> > >>
> > >> *From: *Pablo Estrada <pa...@google.com>
> > >> *Date: *Tue, May 14, 2019 at 1:27 PM
> > >> *To: *dev
> > >> *Cc: *Lars Francke
> > >>
> > >> +1 on merging.
> > >>>
> > >>> *From: *Reuven Lax <re...@google.com>
> > >>> *Date: *Tue, May 14, 2019 at 1:23 PM
> > >>> *To: *dev
> > >>> *Cc: *Lars Francke
> > >>>
> > >>> I've been playing around with this that past day .or two, and it's
> > >>>> great! I'm inclined to merge this PR (if nobody objects) so that
> others in
> > >>>> the community can contribute more training katas.
> > >>>>
> > >>>> Reuven
> > >>>>
> > >>>> *From: *Ismaël Mejía <ie...@gmail.com>
> > >>>> *Date: *Tue, Apr 23, 2019 at 6:43 AM
> > >>>> *To: *Lars Francke
> > >>>> *Cc: * <de...@beam.apache.org>
> > >>>>
> > >>>> Thanks for answering Lars,
> > >>>>>
> > >>>>> The 'interesting' part is that the tutorial has a full IDE
> integrated
> > >>>>> experience based on the Jetbrains edu platform [1]. So maybe
> > >>>>> interesting to see if it could make sense to have projects like
> this
> > >>>>> in the new trainings incubator project or if they became too
> platform
> > >>>>> constrained.
> > >>>>>
> > >>>>> This contribution is valuable for Beam but the community may decide
> > >>>>> that it makes sense for it to live at some moment at the trainings
> > >>>>> project. I suppose also Henry could be interested in taking a look
> at
> > >>>>> this [2].
> > >>>>>
> > >>>>> [1] https://www.jetbrains.com/education/
> > >>>>> [2] https://incubator.apache.org/clutch/training.html
> > >>>>>
> > >>>>> On Tue, Apr 23, 2019 at 3:00 PM Lars Francke <
> lars.francke@gmail.com>
> > >>>>> wrote:
> > >>>>> >
> > >>>>> > Thanks Ismaël.
> > >>>>> >
> > >>>>> > I must admit I'm a tad confused. What has JetBrains got to do
> with
> > >>>>> this?
> > >>>>> > This looks pretty cool and specific to Beam though, or is this
> more
> > >>>>> generic?
> > >>>>> > But yeah something along those lines could be interesting for
> > >>>>> hands-on type things in training.
> > >>>>> >
> > >>>>> > On Fri, Apr 19, 2019 at 12:10 PM Ismaël Mejía <iemejia@gmail.com
> >
> > >>>>> wrote:
> > >>>>> >>
> > >>>>> >> +lars.francke@gmail.com who is in the Apache training project
> and
> > >>>>> may
> > >>>>> >> be interested in this one or at least the JetBrains like
> approach.
> > >>>>> >>
> > >>>>> >> On Fri, Apr 19, 2019 at 12:01 PM Ismaël Mejía <
> iemejia@gmail.com>
> > >>>>> wrote:
> > >>>>> >> >
> > >>>>> >> > This looks great, nice for bringing this to the project Henry!
> > >>>>> >> >
> > >>>>> >> > On Fri, Apr 19, 2019 at 10:53 AM hsuryawirawan@google.com
> > >>>>> >> > <hs...@google.com> wrote:
> > >>>>> >> > >
> > >>>>> >> > > Thanks Altay.
> > >>>>> >> > > I'll create it under "learning/" first as this is not
> exactly
> > >>>>> example.
> > >>>>> >> > > Please do let me know if it's not the right place.
> > >>>>> >> > >
> > >>>>> >> > > On 2019/04/18 22:49:47, Ahmet Altay <al...@google.com>
> wrote:
> > >>>>> >> > > > This looks great.
> > >>>>> >> > > >
> > >>>>> >> > > > +David Cavazos <dc...@google.com> was working on
> > >>>>> interactive colab based
> > >>>>> >> > > > examples (https://github.com/apache/beam/pull/7679)
> perhaps
> > >>>>> we can have a
> > >>>>> >> > > > shared place for these two similar things.
> > >>>>> >> > > >
> > >>>>> >> > >
> > >>>>>
> > >>>>
> > >
> > > --
> > >
> > > This email may be confidential and privileged. If you received this
> > > communication by mistake, please don't forward it to anyone else,
> please
> > > erase all copies and attachments, and please let me know that it has
> gone
> > > to the wrong person.
> > >
> > > The above terms reflect a potential business arrangement, are provided
> > > solely as a basis for further discussion, and are not intended to be
> and do
> > > not constitute a legally binding obligation. No legally binding
> obligations
> > > will be created, implied, or inferred until an agreement in final form
> is
> > > executed in writing by all parties involved.
> > >
> >
>
Re: Contributing Beam Kata (Java & Python)
Posted by .
Thanks for merging it Reuven!
Quick question, would it be useful if we write a blog post on the Kata so that we can build more awareness for people to try out?
I've also uploaded the course to Stepik which has seamless integration within the IDE for people to easily start the course.
On 2019/05/14 20:35:07, Reuven Lax <re...@google.com> wrote:
> Merged
>
> *From: *Reza Rokni <re...@google.com>
> *Date: *Tue, May 14, 2019 at 1:29 PM
> *To: * <de...@beam.apache.org>
> *Cc: *Lars Francke
>
> +1 :-)
> >
> > *From: *Lukasz Cwik <lc...@google.com>
> > *Date: *Wed, 15 May 2019 at 04:29
> > *To: *dev
> > *Cc: *Lars Francke
> >
> > +1
> >>
> >> *From: *Pablo Estrada <pa...@google.com>
> >> *Date: *Tue, May 14, 2019 at 1:27 PM
> >> *To: *dev
> >> *Cc: *Lars Francke
> >>
> >> +1 on merging.
> >>>
> >>> *From: *Reuven Lax <re...@google.com>
> >>> *Date: *Tue, May 14, 2019 at 1:23 PM
> >>> *To: *dev
> >>> *Cc: *Lars Francke
> >>>
> >>> I've been playing around with this that past day .or two, and it's
> >>>> great! I'm inclined to merge this PR (if nobody objects) so that others in
> >>>> the community can contribute more training katas.
> >>>>
> >>>> Reuven
> >>>>
> >>>> *From: *Ismaël Mejía <ie...@gmail.com>
> >>>> *Date: *Tue, Apr 23, 2019 at 6:43 AM
> >>>> *To: *Lars Francke
> >>>> *Cc: * <de...@beam.apache.org>
> >>>>
> >>>> Thanks for answering Lars,
> >>>>>
> >>>>> The 'interesting' part is that the tutorial has a full IDE integrated
> >>>>> experience based on the Jetbrains edu platform [1]. So maybe
> >>>>> interesting to see if it could make sense to have projects like this
> >>>>> in the new trainings incubator project or if they became too platform
> >>>>> constrained.
> >>>>>
> >>>>> This contribution is valuable for Beam but the community may decide
> >>>>> that it makes sense for it to live at some moment at the trainings
> >>>>> project. I suppose also Henry could be interested in taking a look at
> >>>>> this [2].
> >>>>>
> >>>>> [1] https://www.jetbrains.com/education/
> >>>>> [2] https://incubator.apache.org/clutch/training.html
> >>>>>
> >>>>> On Tue, Apr 23, 2019 at 3:00 PM Lars Francke <la...@gmail.com>
> >>>>> wrote:
> >>>>> >
> >>>>> > Thanks Ismaël.
> >>>>> >
> >>>>> > I must admit I'm a tad confused. What has JetBrains got to do with
> >>>>> this?
> >>>>> > This looks pretty cool and specific to Beam though, or is this more
> >>>>> generic?
> >>>>> > But yeah something along those lines could be interesting for
> >>>>> hands-on type things in training.
> >>>>> >
> >>>>> > On Fri, Apr 19, 2019 at 12:10 PM Ismaël Mejía <ie...@gmail.com>
> >>>>> wrote:
> >>>>> >>
> >>>>> >> +lars.francke@gmail.com who is in the Apache training project and
> >>>>> may
> >>>>> >> be interested in this one or at least the JetBrains like approach.
> >>>>> >>
> >>>>> >> On Fri, Apr 19, 2019 at 12:01 PM Ismaël Mejía <ie...@gmail.com>
> >>>>> wrote:
> >>>>> >> >
> >>>>> >> > This looks great, nice for bringing this to the project Henry!
> >>>>> >> >
> >>>>> >> > On Fri, Apr 19, 2019 at 10:53 AM hsuryawirawan@google.com
> >>>>> >> > <hs...@google.com> wrote:
> >>>>> >> > >
> >>>>> >> > > Thanks Altay.
> >>>>> >> > > I'll create it under "learning/" first as this is not exactly
> >>>>> example.
> >>>>> >> > > Please do let me know if it's not the right place.
> >>>>> >> > >
> >>>>> >> > > On 2019/04/18 22:49:47, Ahmet Altay <al...@google.com> wrote:
> >>>>> >> > > > This looks great.
> >>>>> >> > > >
> >>>>> >> > > > +David Cavazos <dc...@google.com> was working on
> >>>>> interactive colab based
> >>>>> >> > > > examples (https://github.com/apache/beam/pull/7679) perhaps
> >>>>> we can have a
> >>>>> >> > > > shared place for these two similar things.
> >>>>> >> > > >
> >>>>> >> > >
> >>>>>
> >>>>
> >
> > --
> >
> > This email may be confidential and privileged. If you received this
> > communication by mistake, please don't forward it to anyone else, please
> > erase all copies and attachments, and please let me know that it has gone
> > to the wrong person.
> >
> > The above terms reflect a potential business arrangement, are provided
> > solely as a basis for further discussion, and are not intended to be and do
> > not constitute a legally binding obligation. No legally binding obligations
> > will be created, implied, or inferred until an agreement in final form is
> > executed in writing by all parties involved.
> >
>
Re: Contributing Beam Kata (Java & Python)
Posted by Reuven Lax <re...@google.com>.
Merged
*From: *Reza Rokni <re...@google.com>
*Date: *Tue, May 14, 2019 at 1:29 PM
*To: * <de...@beam.apache.org>
*Cc: *Lars Francke
+1 :-)
>
> *From: *Lukasz Cwik <lc...@google.com>
> *Date: *Wed, 15 May 2019 at 04:29
> *To: *dev
> *Cc: *Lars Francke
>
> +1
>>
>> *From: *Pablo Estrada <pa...@google.com>
>> *Date: *Tue, May 14, 2019 at 1:27 PM
>> *To: *dev
>> *Cc: *Lars Francke
>>
>> +1 on merging.
>>>
>>> *From: *Reuven Lax <re...@google.com>
>>> *Date: *Tue, May 14, 2019 at 1:23 PM
>>> *To: *dev
>>> *Cc: *Lars Francke
>>>
>>> I've been playing around with this that past day .or two, and it's
>>>> great! I'm inclined to merge this PR (if nobody objects) so that others in
>>>> the community can contribute more training katas.
>>>>
>>>> Reuven
>>>>
>>>> *From: *Ismaël Mejía <ie...@gmail.com>
>>>> *Date: *Tue, Apr 23, 2019 at 6:43 AM
>>>> *To: *Lars Francke
>>>> *Cc: * <de...@beam.apache.org>
>>>>
>>>> Thanks for answering Lars,
>>>>>
>>>>> The 'interesting' part is that the tutorial has a full IDE integrated
>>>>> experience based on the Jetbrains edu platform [1]. So maybe
>>>>> interesting to see if it could make sense to have projects like this
>>>>> in the new trainings incubator project or if they became too platform
>>>>> constrained.
>>>>>
>>>>> This contribution is valuable for Beam but the community may decide
>>>>> that it makes sense for it to live at some moment at the trainings
>>>>> project. I suppose also Henry could be interested in taking a look at
>>>>> this [2].
>>>>>
>>>>> [1] https://www.jetbrains.com/education/
>>>>> [2] https://incubator.apache.org/clutch/training.html
>>>>>
>>>>> On Tue, Apr 23, 2019 at 3:00 PM Lars Francke <la...@gmail.com>
>>>>> wrote:
>>>>> >
>>>>> > Thanks Ismaël.
>>>>> >
>>>>> > I must admit I'm a tad confused. What has JetBrains got to do with
>>>>> this?
>>>>> > This looks pretty cool and specific to Beam though, or is this more
>>>>> generic?
>>>>> > But yeah something along those lines could be interesting for
>>>>> hands-on type things in training.
>>>>> >
>>>>> > On Fri, Apr 19, 2019 at 12:10 PM Ismaël Mejía <ie...@gmail.com>
>>>>> wrote:
>>>>> >>
>>>>> >> +lars.francke@gmail.com who is in the Apache training project and
>>>>> may
>>>>> >> be interested in this one or at least the JetBrains like approach.
>>>>> >>
>>>>> >> On Fri, Apr 19, 2019 at 12:01 PM Ismaël Mejía <ie...@gmail.com>
>>>>> wrote:
>>>>> >> >
>>>>> >> > This looks great, nice for bringing this to the project Henry!
>>>>> >> >
>>>>> >> > On Fri, Apr 19, 2019 at 10:53 AM hsuryawirawan@google.com
>>>>> >> > <hs...@google.com> wrote:
>>>>> >> > >
>>>>> >> > > Thanks Altay.
>>>>> >> > > I'll create it under "learning/" first as this is not exactly
>>>>> example.
>>>>> >> > > Please do let me know if it's not the right place.
>>>>> >> > >
>>>>> >> > > On 2019/04/18 22:49:47, Ahmet Altay <al...@google.com> wrote:
>>>>> >> > > > This looks great.
>>>>> >> > > >
>>>>> >> > > > +David Cavazos <dc...@google.com> was working on
>>>>> interactive colab based
>>>>> >> > > > examples (https://github.com/apache/beam/pull/7679) perhaps
>>>>> we can have a
>>>>> >> > > > shared place for these two similar things.
>>>>> >> > > >
>>>>> >> > >
>>>>>
>>>>
>
> --
>
> This email may be confidential and privileged. If you received this
> communication by mistake, please don't forward it to anyone else, please
> erase all copies and attachments, and please let me know that it has gone
> to the wrong person.
>
> The above terms reflect a potential business arrangement, are provided
> solely as a basis for further discussion, and are not intended to be and do
> not constitute a legally binding obligation. No legally binding obligations
> will be created, implied, or inferred until an agreement in final form is
> executed in writing by all parties involved.
>
Re: Contributing Beam Kata (Java & Python)
Posted by Reza Rokni <re...@google.com>.
+1 :-)
*From: *Lukasz Cwik <lc...@google.com>
*Date: *Wed, 15 May 2019 at 04:29
*To: *dev
*Cc: *Lars Francke
+1
>
> *From: *Pablo Estrada <pa...@google.com>
> *Date: *Tue, May 14, 2019 at 1:27 PM
> *To: *dev
> *Cc: *Lars Francke
>
> +1 on merging.
>>
>> *From: *Reuven Lax <re...@google.com>
>> *Date: *Tue, May 14, 2019 at 1:23 PM
>> *To: *dev
>> *Cc: *Lars Francke
>>
>> I've been playing around with this that past day .or two, and it's great!
>>> I'm inclined to merge this PR (if nobody objects) so that others in the
>>> community can contribute more training katas.
>>>
>>> Reuven
>>>
>>> *From: *Ismaël Mejía <ie...@gmail.com>
>>> *Date: *Tue, Apr 23, 2019 at 6:43 AM
>>> *To: *Lars Francke
>>> *Cc: * <de...@beam.apache.org>
>>>
>>> Thanks for answering Lars,
>>>>
>>>> The 'interesting' part is that the tutorial has a full IDE integrated
>>>> experience based on the Jetbrains edu platform [1]. So maybe
>>>> interesting to see if it could make sense to have projects like this
>>>> in the new trainings incubator project or if they became too platform
>>>> constrained.
>>>>
>>>> This contribution is valuable for Beam but the community may decide
>>>> that it makes sense for it to live at some moment at the trainings
>>>> project. I suppose also Henry could be interested in taking a look at
>>>> this [2].
>>>>
>>>> [1] https://www.jetbrains.com/education/
>>>> [2] https://incubator.apache.org/clutch/training.html
>>>>
>>>> On Tue, Apr 23, 2019 at 3:00 PM Lars Francke <la...@gmail.com>
>>>> wrote:
>>>> >
>>>> > Thanks Ismaël.
>>>> >
>>>> > I must admit I'm a tad confused. What has JetBrains got to do with
>>>> this?
>>>> > This looks pretty cool and specific to Beam though, or is this more
>>>> generic?
>>>> > But yeah something along those lines could be interesting for
>>>> hands-on type things in training.
>>>> >
>>>> > On Fri, Apr 19, 2019 at 12:10 PM Ismaël Mejía <ie...@gmail.com>
>>>> wrote:
>>>> >>
>>>> >> +lars.francke@gmail.com who is in the Apache training project and
>>>> may
>>>> >> be interested in this one or at least the JetBrains like approach.
>>>> >>
>>>> >> On Fri, Apr 19, 2019 at 12:01 PM Ismaël Mejía <ie...@gmail.com>
>>>> wrote:
>>>> >> >
>>>> >> > This looks great, nice for bringing this to the project Henry!
>>>> >> >
>>>> >> > On Fri, Apr 19, 2019 at 10:53 AM hsuryawirawan@google.com
>>>> >> > <hs...@google.com> wrote:
>>>> >> > >
>>>> >> > > Thanks Altay.
>>>> >> > > I'll create it under "learning/" first as this is not exactly
>>>> example.
>>>> >> > > Please do let me know if it's not the right place.
>>>> >> > >
>>>> >> > > On 2019/04/18 22:49:47, Ahmet Altay <al...@google.com> wrote:
>>>> >> > > > This looks great.
>>>> >> > > >
>>>> >> > > > +David Cavazos <dc...@google.com> was working on
>>>> interactive colab based
>>>> >> > > > examples (https://github.com/apache/beam/pull/7679) perhaps
>>>> we can have a
>>>> >> > > > shared place for these two similar things.
>>>> >> > > >
>>>> >> > >
>>>>
>>>
--
This email may be confidential and privileged. If you received this
communication by mistake, please don't forward it to anyone else, please
erase all copies and attachments, and please let me know that it has gone
to the wrong person.
The above terms reflect a potential business arrangement, are provided
solely as a basis for further discussion, and are not intended to be and do
not constitute a legally binding obligation. No legally binding obligations
will be created, implied, or inferred until an agreement in final form is
executed in writing by all parties involved.
Re: Contributing Beam Kata (Java & Python)
Posted by Lukasz Cwik <lc...@google.com>.
+1
*From: *Pablo Estrada <pa...@google.com>
*Date: *Tue, May 14, 2019 at 1:27 PM
*To: *dev
*Cc: *Lars Francke
+1 on merging.
>
> *From: *Reuven Lax <re...@google.com>
> *Date: *Tue, May 14, 2019 at 1:23 PM
> *To: *dev
> *Cc: *Lars Francke
>
> I've been playing around with this that past day .or two, and it's great!
>> I'm inclined to merge this PR (if nobody objects) so that others in the
>> community can contribute more training katas.
>>
>> Reuven
>>
>> *From: *Ismaël Mejía <ie...@gmail.com>
>> *Date: *Tue, Apr 23, 2019 at 6:43 AM
>> *To: *Lars Francke
>> *Cc: * <de...@beam.apache.org>
>>
>> Thanks for answering Lars,
>>>
>>> The 'interesting' part is that the tutorial has a full IDE integrated
>>> experience based on the Jetbrains edu platform [1]. So maybe
>>> interesting to see if it could make sense to have projects like this
>>> in the new trainings incubator project or if they became too platform
>>> constrained.
>>>
>>> This contribution is valuable for Beam but the community may decide
>>> that it makes sense for it to live at some moment at the trainings
>>> project. I suppose also Henry could be interested in taking a look at
>>> this [2].
>>>
>>> [1] https://www.jetbrains.com/education/
>>> [2] https://incubator.apache.org/clutch/training.html
>>>
>>> On Tue, Apr 23, 2019 at 3:00 PM Lars Francke <la...@gmail.com>
>>> wrote:
>>> >
>>> > Thanks Ismaël.
>>> >
>>> > I must admit I'm a tad confused. What has JetBrains got to do with
>>> this?
>>> > This looks pretty cool and specific to Beam though, or is this more
>>> generic?
>>> > But yeah something along those lines could be interesting for hands-on
>>> type things in training.
>>> >
>>> > On Fri, Apr 19, 2019 at 12:10 PM Ismaël Mejía <ie...@gmail.com>
>>> wrote:
>>> >>
>>> >> +lars.francke@gmail.com who is in the Apache training project and may
>>> >> be interested in this one or at least the JetBrains like approach.
>>> >>
>>> >> On Fri, Apr 19, 2019 at 12:01 PM Ismaël Mejía <ie...@gmail.com>
>>> wrote:
>>> >> >
>>> >> > This looks great, nice for bringing this to the project Henry!
>>> >> >
>>> >> > On Fri, Apr 19, 2019 at 10:53 AM hsuryawirawan@google.com
>>> >> > <hs...@google.com> wrote:
>>> >> > >
>>> >> > > Thanks Altay.
>>> >> > > I'll create it under "learning/" first as this is not exactly
>>> example.
>>> >> > > Please do let me know if it's not the right place.
>>> >> > >
>>> >> > > On 2019/04/18 22:49:47, Ahmet Altay <al...@google.com> wrote:
>>> >> > > > This looks great.
>>> >> > > >
>>> >> > > > +David Cavazos <dc...@google.com> was working on
>>> interactive colab based
>>> >> > > > examples (https://github.com/apache/beam/pull/7679) perhaps we
>>> can have a
>>> >> > > > shared place for these two similar things.
>>> >> > > >
>>> >> > >
>>>
>>
Re: Contributing Beam Kata (Java & Python)
Posted by Pablo Estrada <pa...@google.com>.
+1 on merging.
*From: *Reuven Lax <re...@google.com>
*Date: *Tue, May 14, 2019 at 1:23 PM
*To: *dev
*Cc: *Lars Francke
I've been playing around with this that past day .or two, and it's great!
> I'm inclined to merge this PR (if nobody objects) so that others in the
> community can contribute more training katas.
>
> Reuven
>
> *From: *Ismaël Mejía <ie...@gmail.com>
> *Date: *Tue, Apr 23, 2019 at 6:43 AM
> *To: *Lars Francke
> *Cc: * <de...@beam.apache.org>
>
> Thanks for answering Lars,
>>
>> The 'interesting' part is that the tutorial has a full IDE integrated
>> experience based on the Jetbrains edu platform [1]. So maybe
>> interesting to see if it could make sense to have projects like this
>> in the new trainings incubator project or if they became too platform
>> constrained.
>>
>> This contribution is valuable for Beam but the community may decide
>> that it makes sense for it to live at some moment at the trainings
>> project. I suppose also Henry could be interested in taking a look at
>> this [2].
>>
>> [1] https://www.jetbrains.com/education/
>> [2] https://incubator.apache.org/clutch/training.html
>>
>> On Tue, Apr 23, 2019 at 3:00 PM Lars Francke <la...@gmail.com>
>> wrote:
>> >
>> > Thanks Ismaël.
>> >
>> > I must admit I'm a tad confused. What has JetBrains got to do with this?
>> > This looks pretty cool and specific to Beam though, or is this more
>> generic?
>> > But yeah something along those lines could be interesting for hands-on
>> type things in training.
>> >
>> > On Fri, Apr 19, 2019 at 12:10 PM Ismaël Mejía <ie...@gmail.com>
>> wrote:
>> >>
>> >> +lars.francke@gmail.com who is in the Apache training project and may
>> >> be interested in this one or at least the JetBrains like approach.
>> >>
>> >> On Fri, Apr 19, 2019 at 12:01 PM Ismaël Mejía <ie...@gmail.com>
>> wrote:
>> >> >
>> >> > This looks great, nice for bringing this to the project Henry!
>> >> >
>> >> > On Fri, Apr 19, 2019 at 10:53 AM hsuryawirawan@google.com
>> >> > <hs...@google.com> wrote:
>> >> > >
>> >> > > Thanks Altay.
>> >> > > I'll create it under "learning/" first as this is not exactly
>> example.
>> >> > > Please do let me know if it's not the right place.
>> >> > >
>> >> > > On 2019/04/18 22:49:47, Ahmet Altay <al...@google.com> wrote:
>> >> > > > This looks great.
>> >> > > >
>> >> > > > +David Cavazos <dc...@google.com> was working on interactive
>> colab based
>> >> > > > examples (https://github.com/apache/beam/pull/7679) perhaps we
>> can have a
>> >> > > > shared place for these two similar things.
>> >> > > >
>> >> > >
>>
>
Re: Contributing Beam Kata (Java & Python)
Posted by Reuven Lax <re...@google.com>.
I've been playing around with this that past day .or two, and it's great!
I'm inclined to merge this PR (if nobody objects) so that others in the
community can contribute more training katas.
Reuven
*From: *Ismaël Mejía <ie...@gmail.com>
*Date: *Tue, Apr 23, 2019 at 6:43 AM
*To: *Lars Francke
*Cc: * <de...@beam.apache.org>
Thanks for answering Lars,
>
> The 'interesting' part is that the tutorial has a full IDE integrated
> experience based on the Jetbrains edu platform [1]. So maybe
> interesting to see if it could make sense to have projects like this
> in the new trainings incubator project or if they became too platform
> constrained.
>
> This contribution is valuable for Beam but the community may decide
> that it makes sense for it to live at some moment at the trainings
> project. I suppose also Henry could be interested in taking a look at
> this [2].
>
> [1] https://www.jetbrains.com/education/
> [2] https://incubator.apache.org/clutch/training.html
>
> On Tue, Apr 23, 2019 at 3:00 PM Lars Francke <la...@gmail.com>
> wrote:
> >
> > Thanks Ismaël.
> >
> > I must admit I'm a tad confused. What has JetBrains got to do with this?
> > This looks pretty cool and specific to Beam though, or is this more
> generic?
> > But yeah something along those lines could be interesting for hands-on
> type things in training.
> >
> > On Fri, Apr 19, 2019 at 12:10 PM Ismaël Mejía <ie...@gmail.com> wrote:
> >>
> >> +lars.francke@gmail.com who is in the Apache training project and may
> >> be interested in this one or at least the JetBrains like approach.
> >>
> >> On Fri, Apr 19, 2019 at 12:01 PM Ismaël Mejía <ie...@gmail.com>
> wrote:
> >> >
> >> > This looks great, nice for bringing this to the project Henry!
> >> >
> >> > On Fri, Apr 19, 2019 at 10:53 AM hsuryawirawan@google.com
> >> > <hs...@google.com> wrote:
> >> > >
> >> > > Thanks Altay.
> >> > > I'll create it under "learning/" first as this is not exactly
> example.
> >> > > Please do let me know if it's not the right place.
> >> > >
> >> > > On 2019/04/18 22:49:47, Ahmet Altay <al...@google.com> wrote:
> >> > > > This looks great.
> >> > > >
> >> > > > +David Cavazos <dc...@google.com> was working on interactive
> colab based
> >> > > > examples (https://github.com/apache/beam/pull/7679) perhaps we
> can have a
> >> > > > shared place for these two similar things.
> >> > > >
> >> > >
>
Re: Contributing Beam Kata (Java & Python)
Posted by Ismaël Mejía <ie...@gmail.com>.
Thanks for answering Lars,
The 'interesting' part is that the tutorial has a full IDE integrated
experience based on the Jetbrains edu platform [1]. So maybe
interesting to see if it could make sense to have projects like this
in the new trainings incubator project or if they became too platform
constrained.
This contribution is valuable for Beam but the community may decide
that it makes sense for it to live at some moment at the trainings
project. I suppose also Henry could be interested in taking a look at
this [2].
[1] https://www.jetbrains.com/education/
[2] https://incubator.apache.org/clutch/training.html
On Tue, Apr 23, 2019 at 3:00 PM Lars Francke <la...@gmail.com> wrote:
>
> Thanks Ismaël.
>
> I must admit I'm a tad confused. What has JetBrains got to do with this?
> This looks pretty cool and specific to Beam though, or is this more generic?
> But yeah something along those lines could be interesting for hands-on type things in training.
>
> On Fri, Apr 19, 2019 at 12:10 PM Ismaël Mejía <ie...@gmail.com> wrote:
>>
>> +lars.francke@gmail.com who is in the Apache training project and may
>> be interested in this one or at least the JetBrains like approach.
>>
>> On Fri, Apr 19, 2019 at 12:01 PM Ismaël Mejía <ie...@gmail.com> wrote:
>> >
>> > This looks great, nice for bringing this to the project Henry!
>> >
>> > On Fri, Apr 19, 2019 at 10:53 AM hsuryawirawan@google.com
>> > <hs...@google.com> wrote:
>> > >
>> > > Thanks Altay.
>> > > I'll create it under "learning/" first as this is not exactly example.
>> > > Please do let me know if it's not the right place.
>> > >
>> > > On 2019/04/18 22:49:47, Ahmet Altay <al...@google.com> wrote:
>> > > > This looks great.
>> > > >
>> > > > +David Cavazos <dc...@google.com> was working on interactive colab based
>> > > > examples (https://github.com/apache/beam/pull/7679) perhaps we can have a
>> > > > shared place for these two similar things.
>> > > >
>> > >
Re: Contributing Beam Kata (Java & Python)
Posted by Lars Francke <la...@gmail.com>.
Thanks Ismaël.
I must admit I'm a tad confused. What has JetBrains got to do with this?
This looks pretty cool and specific to Beam though, or is this more generic?
But yeah something along those lines could be interesting for hands-on type
things in training.
On Fri, Apr 19, 2019 at 12:10 PM Ismaël Mejía <ie...@gmail.com> wrote:
> +lars.francke@gmail.com who is in the Apache training project and may
> be interested in this one or at least the JetBrains like approach.
>
> On Fri, Apr 19, 2019 at 12:01 PM Ismaël Mejía <ie...@gmail.com> wrote:
> >
> > This looks great, nice for bringing this to the project Henry!
> >
> > On Fri, Apr 19, 2019 at 10:53 AM hsuryawirawan@google.com
> > <hs...@google.com> wrote:
> > >
> > > Thanks Altay.
> > > I'll create it under "learning/" first as this is not exactly example.
> > > Please do let me know if it's not the right place.
> > >
> > > On 2019/04/18 22:49:47, Ahmet Altay <al...@google.com> wrote:
> > > > This looks great.
> > > >
> > > > +David Cavazos <dc...@google.com> was working on interactive
> colab based
> > > > examples (https://github.com/apache/beam/pull/7679) perhaps we can
> have a
> > > > shared place for these two similar things.
> > > >
> > >
>
Re: Contributing Beam Kata (Java & Python)
Posted by Ismaël Mejía <ie...@gmail.com>.
+lars.francke@gmail.com who is in the Apache training project and may
be interested in this one or at least the JetBrains like approach.
On Fri, Apr 19, 2019 at 12:01 PM Ismaël Mejía <ie...@gmail.com> wrote:
>
> This looks great, nice for bringing this to the project Henry!
>
> On Fri, Apr 19, 2019 at 10:53 AM hsuryawirawan@google.com
> <hs...@google.com> wrote:
> >
> > Thanks Altay.
> > I'll create it under "learning/" first as this is not exactly example.
> > Please do let me know if it's not the right place.
> >
> > On 2019/04/18 22:49:47, Ahmet Altay <al...@google.com> wrote:
> > > This looks great.
> > >
> > > +David Cavazos <dc...@google.com> was working on interactive colab based
> > > examples (https://github.com/apache/beam/pull/7679) perhaps we can have a
> > > shared place for these two similar things.
> > >
> >
Re: Contributing Beam Kata (Java & Python)
Posted by Ismaël Mejía <ie...@gmail.com>.
This looks great, nice for bringing this to the project Henry!
On Fri, Apr 19, 2019 at 10:53 AM hsuryawirawan@google.com
<hs...@google.com> wrote:
>
> Thanks Altay.
> I'll create it under "learning/" first as this is not exactly example.
> Please do let me know if it's not the right place.
>
> On 2019/04/18 22:49:47, Ahmet Altay <al...@google.com> wrote:
> > This looks great.
> >
> > +David Cavazos <dc...@google.com> was working on interactive colab based
> > examples (https://github.com/apache/beam/pull/7679) perhaps we can have a
> > shared place for these two similar things.
> >
>
Re: Contributing Beam Kata (Java & Python)
Posted by .
Thanks Altay.
I'll create it under "learning/" first as this is not exactly example.
Please do let me know if it's not the right place.
On 2019/04/18 22:49:47, Ahmet Altay <al...@google.com> wrote:
> This looks great.
>
> +David Cavazos <dc...@google.com> was working on interactive colab based
> examples (https://github.com/apache/beam/pull/7679) perhaps we can have a
> shared place for these two similar things.
>
Re: Contributing Beam Kata (Java & Python)
Posted by Ahmet Altay <al...@google.com>.
This looks great.
+David Cavazos <dc...@google.com> was working on interactive colab based
examples (https://github.com/apache/beam/pull/7679) perhaps we can have a
shared place for these two similar things.
On Thu, Apr 18, 2019 at 2:50 PM Kenneth Knowles <ke...@apache.org> wrote:
> Great! By the way it might fit in with the newly incubating training
> project [1]. Do you go into specifics on using different runners at all?
>
> Kenn
>
> [1] https://wiki.apache.org/incubator/TrainingProposal
>
> On Thu, Apr 18, 2019 at 2:25 PM Anton Kedin <ke...@google.com> wrote:
>
>> +1, looks really great. And I agree with Pablo, something along the lines
>> of `learning/...` subdirectory feels appropriate.
>>
>> Regards,
>> Anton
>>
>> On Thu, Apr 18, 2019 at 2:21 PM Lukasz Cwik <lc...@google.com> wrote:
>>
>>> Also agree that this is really nice. Is there a place where we can try
>>> out what you have created so far?
>>>
>>> Opening a PR with what you have created so far may be the easiest way to
>>> convey what your thinking would make sense.
>>>
>>>
>>>
>>>
>>> On Thu, Apr 18, 2019 at 1:41 PM Pablo Estrada <pa...@google.com>
>>> wrote:
>>>
>>>> Hi Henry!
>>>> this seems quite nice! For my information, what does the file structure
>>>> look like for something like this? Also, maybe it's not the intended way of
>>>> introducing it, but perhaps, could you share a video showing how one would
>>>> try out the first kata?
>>>>
>>>> Something that comes to mind for having this in Beam is having a
>>>> `learning/katas/` subdirectory in the repository, with a README; and pehaps
>>>> a section on the website for people to try this out.
>>>>
>>>> Another question that comes to mind: Is it possible to structure it so
>>>> that katas for other languages can be added?
>>>>
>>>> Thoughts from others?
>>>> Best
>>>> -P.
>>>>
>>>> On Thu, Apr 18, 2019 at 1:20 PM Henry Suryawirawan <
>>>> hsuryawirawan@google.com> wrote:
>>>>
>>>>> Hi Beam Devs,
>>>>>
>>>>> I have been working on what I call Beam Kata, a set of interactive
>>>>> Beam coding exercises (i.e. code kata) that aids in learning Beam
>>>>> programming hands-on. It is built based on JetBrains Educational Products (
>>>>> https://www.jetbrains.com/education/).
>>>>> The objective of Beam Kata is to provide a structured hands-on
>>>>> learning experience for people to learn about Beam and its SDK by solving
>>>>> focused mini problems with gradually increasing complexity.
>>>>> I have used the Kata to conduct Beam workshops, teaching the
>>>>> participants on how to code in Beam, and so far the feedback has been great.
>>>>> They particularly like the hands-on and how it is closely integrated
>>>>> with the IDE, giving the best developer experience, especially for
>>>>> IntelliJ/PyCharm developers.
>>>>>
>>>>> I would like to contribute this to the Beam repository and would like
>>>>> to check with you whether this is something that would be useful as an
>>>>> addition.
>>>>> If so, can you advise me in which directory I should commit my work?
>>>>> If you need further information, please do let me know.
>>>>>
>>>>> As of the moment, I have created lessons for both Java & Python SDK.
>>>>> The current lessons are:
>>>>> # Introduction
>>>>> * Hello Beam
>>>>>
>>>>> # Core Transforms
>>>>> * Map
>>>>> ** ParDo
>>>>> ** ParDo OneToMany
>>>>> ** MapElements / Map
>>>>> ** FlatMapElements / FlatMap
>>>>> * GroupByKey
>>>>> * CoGroupByKey
>>>>> * Combine
>>>>> ** Simple Function
>>>>> ** CombineFn
>>>>> ** BinaryCombineFn
>>>>> ** Combine PerKey
>>>>> * Flatten
>>>>> * Partition
>>>>>
>>>>> # Common Transforms
>>>>> * Filter
>>>>> ** ParDo
>>>>> ** Filter
>>>>> * Aggregation
>>>>> ** Count
>>>>> ** Sum
>>>>> ** Avg
>>>>> ** Min / Smallest
>>>>> ** Max / Largest
>>>>>
>>>>> I have plans to add more lessons to make it more complete, covering
>>>>> more Beam fundamentals, e.g. composite transform, branching (multiple
>>>>> transforms, side output), merging (side input), I/O, windowing, streaming,
>>>>> examples (word count, etc), and possibly use case patterns.
>>>>>
>>>>> Please find some screenshots on how it looks like for a student:
>>>>> [image: image.png]
>>>>>
>>>>> [image: image.png]
>>>>>
>>>>> Regards,
>>>>> Henry Suryawirawan
>>>>>
>>>>
Re: Contributing Beam Kata (Java & Python)
Posted by Kenneth Knowles <ke...@apache.org>.
Great! By the way it might fit in with the newly incubating training
project [1]. Do you go into specifics on using different runners at all?
Kenn
[1] https://wiki.apache.org/incubator/TrainingProposal
On Thu, Apr 18, 2019 at 2:25 PM Anton Kedin <ke...@google.com> wrote:
> +1, looks really great. And I agree with Pablo, something along the lines
> of `learning/...` subdirectory feels appropriate.
>
> Regards,
> Anton
>
> On Thu, Apr 18, 2019 at 2:21 PM Lukasz Cwik <lc...@google.com> wrote:
>
>> Also agree that this is really nice. Is there a place where we can try
>> out what you have created so far?
>>
>> Opening a PR with what you have created so far may be the easiest way to
>> convey what your thinking would make sense.
>>
>>
>>
>>
>> On Thu, Apr 18, 2019 at 1:41 PM Pablo Estrada <pa...@google.com> wrote:
>>
>>> Hi Henry!
>>> this seems quite nice! For my information, what does the file structure
>>> look like for something like this? Also, maybe it's not the intended way of
>>> introducing it, but perhaps, could you share a video showing how one would
>>> try out the first kata?
>>>
>>> Something that comes to mind for having this in Beam is having a
>>> `learning/katas/` subdirectory in the repository, with a README; and pehaps
>>> a section on the website for people to try this out.
>>>
>>> Another question that comes to mind: Is it possible to structure it so
>>> that katas for other languages can be added?
>>>
>>> Thoughts from others?
>>> Best
>>> -P.
>>>
>>> On Thu, Apr 18, 2019 at 1:20 PM Henry Suryawirawan <
>>> hsuryawirawan@google.com> wrote:
>>>
>>>> Hi Beam Devs,
>>>>
>>>> I have been working on what I call Beam Kata, a set of interactive Beam
>>>> coding exercises (i.e. code kata) that aids in learning Beam programming
>>>> hands-on. It is built based on JetBrains Educational Products (
>>>> https://www.jetbrains.com/education/).
>>>> The objective of Beam Kata is to provide a structured hands-on learning
>>>> experience for people to learn about Beam and its SDK by solving focused
>>>> mini problems with gradually increasing complexity.
>>>> I have used the Kata to conduct Beam workshops, teaching the
>>>> participants on how to code in Beam, and so far the feedback has been great.
>>>> They particularly like the hands-on and how it is closely integrated
>>>> with the IDE, giving the best developer experience, especially for
>>>> IntelliJ/PyCharm developers.
>>>>
>>>> I would like to contribute this to the Beam repository and would like
>>>> to check with you whether this is something that would be useful as an
>>>> addition.
>>>> If so, can you advise me in which directory I should commit my work?
>>>> If you need further information, please do let me know.
>>>>
>>>> As of the moment, I have created lessons for both Java & Python SDK.
>>>> The current lessons are:
>>>> # Introduction
>>>> * Hello Beam
>>>>
>>>> # Core Transforms
>>>> * Map
>>>> ** ParDo
>>>> ** ParDo OneToMany
>>>> ** MapElements / Map
>>>> ** FlatMapElements / FlatMap
>>>> * GroupByKey
>>>> * CoGroupByKey
>>>> * Combine
>>>> ** Simple Function
>>>> ** CombineFn
>>>> ** BinaryCombineFn
>>>> ** Combine PerKey
>>>> * Flatten
>>>> * Partition
>>>>
>>>> # Common Transforms
>>>> * Filter
>>>> ** ParDo
>>>> ** Filter
>>>> * Aggregation
>>>> ** Count
>>>> ** Sum
>>>> ** Avg
>>>> ** Min / Smallest
>>>> ** Max / Largest
>>>>
>>>> I have plans to add more lessons to make it more complete, covering
>>>> more Beam fundamentals, e.g. composite transform, branching (multiple
>>>> transforms, side output), merging (side input), I/O, windowing, streaming,
>>>> examples (word count, etc), and possibly use case patterns.
>>>>
>>>> Please find some screenshots on how it looks like for a student:
>>>> [image: image.png]
>>>>
>>>> [image: image.png]
>>>>
>>>> Regards,
>>>> Henry Suryawirawan
>>>>
>>>
Re: Contributing Beam Kata (Java & Python)
Posted by Anton Kedin <ke...@google.com>.
+1, looks really great. And I agree with Pablo, something along the lines
of `learning/...` subdirectory feels appropriate.
Regards,
Anton
On Thu, Apr 18, 2019 at 2:21 PM Lukasz Cwik <lc...@google.com> wrote:
> Also agree that this is really nice. Is there a place where we can try out
> what you have created so far?
>
> Opening a PR with what you have created so far may be the easiest way to
> convey what your thinking would make sense.
>
>
>
>
> On Thu, Apr 18, 2019 at 1:41 PM Pablo Estrada <pa...@google.com> wrote:
>
>> Hi Henry!
>> this seems quite nice! For my information, what does the file structure
>> look like for something like this? Also, maybe it's not the intended way of
>> introducing it, but perhaps, could you share a video showing how one would
>> try out the first kata?
>>
>> Something that comes to mind for having this in Beam is having a
>> `learning/katas/` subdirectory in the repository, with a README; and pehaps
>> a section on the website for people to try this out.
>>
>> Another question that comes to mind: Is it possible to structure it so
>> that katas for other languages can be added?
>>
>> Thoughts from others?
>> Best
>> -P.
>>
>> On Thu, Apr 18, 2019 at 1:20 PM Henry Suryawirawan <
>> hsuryawirawan@google.com> wrote:
>>
>>> Hi Beam Devs,
>>>
>>> I have been working on what I call Beam Kata, a set of interactive Beam
>>> coding exercises (i.e. code kata) that aids in learning Beam programming
>>> hands-on. It is built based on JetBrains Educational Products (
>>> https://www.jetbrains.com/education/).
>>> The objective of Beam Kata is to provide a structured hands-on learning
>>> experience for people to learn about Beam and its SDK by solving focused
>>> mini problems with gradually increasing complexity.
>>> I have used the Kata to conduct Beam workshops, teaching the
>>> participants on how to code in Beam, and so far the feedback has been great.
>>> They particularly like the hands-on and how it is closely integrated
>>> with the IDE, giving the best developer experience, especially for
>>> IntelliJ/PyCharm developers.
>>>
>>> I would like to contribute this to the Beam repository and would like to
>>> check with you whether this is something that would be useful as an
>>> addition.
>>> If so, can you advise me in which directory I should commit my work?
>>> If you need further information, please do let me know.
>>>
>>> As of the moment, I have created lessons for both Java & Python SDK.
>>> The current lessons are:
>>> # Introduction
>>> * Hello Beam
>>>
>>> # Core Transforms
>>> * Map
>>> ** ParDo
>>> ** ParDo OneToMany
>>> ** MapElements / Map
>>> ** FlatMapElements / FlatMap
>>> * GroupByKey
>>> * CoGroupByKey
>>> * Combine
>>> ** Simple Function
>>> ** CombineFn
>>> ** BinaryCombineFn
>>> ** Combine PerKey
>>> * Flatten
>>> * Partition
>>>
>>> # Common Transforms
>>> * Filter
>>> ** ParDo
>>> ** Filter
>>> * Aggregation
>>> ** Count
>>> ** Sum
>>> ** Avg
>>> ** Min / Smallest
>>> ** Max / Largest
>>>
>>> I have plans to add more lessons to make it more complete, covering
>>> more Beam fundamentals, e.g. composite transform, branching (multiple
>>> transforms, side output), merging (side input), I/O, windowing, streaming,
>>> examples (word count, etc), and possibly use case patterns.
>>>
>>> Please find some screenshots on how it looks like for a student:
>>> [image: image.png]
>>>
>>> [image: image.png]
>>>
>>> Regards,
>>> Henry Suryawirawan
>>>
>>
Re: Contributing Beam Kata (Java & Python)
Posted by Willem Pienaar <be...@willem.co>.
Awesome work Henry!!
On Fri, Apr 19, 2019, at 7:16 PM, hsuryawirawan@google.com wrote:
> I've created a PR for the Beam Kata.
> https://github.com/apache/beam/pull/8358
>
> If you're interested to experience it, please find the instruction on
> how to set it up on your machine
> https://github.com/apache/beam/pull/8358#issuecomment-484855236
>
> Should you have any issue or further question, please do let me know.
>
>
> Regards,
> Henry
>
> On 2019/04/19 08:44:55, hsuryawirawan@google.com
> <hs...@google.com> wrote:
> > Thanks Lukasz.
> > Yes you can try the kata.
> > I will write a short instruction on how to use it, maybe along with the PR.
> >
> >
> > On 2019/04/18 21:21:28, Lukasz Cwik <lc...@google.com> wrote:
> > > Also agree that this is really nice. Is there a place where we can try out
> > > what you have created so far?
> > >
> > > Opening a PR with what you have created so far may be the easiest way to
> > > convey what your thinking would make sense.
> > >
> >
> >
>
Re: Contributing Beam Kata (Java & Python)
Posted by .
I've created a PR for the Beam Kata.
https://github.com/apache/beam/pull/8358
If you're interested to experience it, please find the instruction on how to set it up on your machine
https://github.com/apache/beam/pull/8358#issuecomment-484855236
Should you have any issue or further question, please do let me know.
Regards,
Henry
On 2019/04/19 08:44:55, hsuryawirawan@google.com <hs...@google.com> wrote:
> Thanks Lukasz.
> Yes you can try the kata.
> I will write a short instruction on how to use it, maybe along with the PR.
>
>
> On 2019/04/18 21:21:28, Lukasz Cwik <lc...@google.com> wrote:
> > Also agree that this is really nice. Is there a place where we can try out
> > what you have created so far?
> >
> > Opening a PR with what you have created so far may be the easiest way to
> > convey what your thinking would make sense.
> >
>
>
Re: Contributing Beam Kata (Java & Python)
Posted by .
Thanks Lukasz.
Yes you can try the kata.
I will write a short instruction on how to use it, maybe along with the PR.
On 2019/04/18 21:21:28, Lukasz Cwik <lc...@google.com> wrote:
> Also agree that this is really nice. Is there a place where we can try out
> what you have created so far?
>
> Opening a PR with what you have created so far may be the easiest way to
> convey what your thinking would make sense.
>
Re: Contributing Beam Kata (Java & Python)
Posted by Lukasz Cwik <lc...@google.com>.
Also agree that this is really nice. Is there a place where we can try out
what you have created so far?
Opening a PR with what you have created so far may be the easiest way to
convey what your thinking would make sense.
On Thu, Apr 18, 2019 at 1:41 PM Pablo Estrada <pa...@google.com> wrote:
> Hi Henry!
> this seems quite nice! For my information, what does the file structure
> look like for something like this? Also, maybe it's not the intended way of
> introducing it, but perhaps, could you share a video showing how one would
> try out the first kata?
>
> Something that comes to mind for having this in Beam is having a
> `learning/katas/` subdirectory in the repository, with a README; and pehaps
> a section on the website for people to try this out.
>
> Another question that comes to mind: Is it possible to structure it so
> that katas for other languages can be added?
>
> Thoughts from others?
> Best
> -P.
>
> On Thu, Apr 18, 2019 at 1:20 PM Henry Suryawirawan <
> hsuryawirawan@google.com> wrote:
>
>> Hi Beam Devs,
>>
>> I have been working on what I call Beam Kata, a set of interactive Beam
>> coding exercises (i.e. code kata) that aids in learning Beam programming
>> hands-on. It is built based on JetBrains Educational Products (
>> https://www.jetbrains.com/education/).
>> The objective of Beam Kata is to provide a structured hands-on learning
>> experience for people to learn about Beam and its SDK by solving focused
>> mini problems with gradually increasing complexity.
>> I have used the Kata to conduct Beam workshops, teaching the participants
>> on how to code in Beam, and so far the feedback has been great.
>> They particularly like the hands-on and how it is closely integrated with
>> the IDE, giving the best developer experience, especially for
>> IntelliJ/PyCharm developers.
>>
>> I would like to contribute this to the Beam repository and would like to
>> check with you whether this is something that would be useful as an
>> addition.
>> If so, can you advise me in which directory I should commit my work?
>> If you need further information, please do let me know.
>>
>> As of the moment, I have created lessons for both Java & Python SDK.
>> The current lessons are:
>> # Introduction
>> * Hello Beam
>>
>> # Core Transforms
>> * Map
>> ** ParDo
>> ** ParDo OneToMany
>> ** MapElements / Map
>> ** FlatMapElements / FlatMap
>> * GroupByKey
>> * CoGroupByKey
>> * Combine
>> ** Simple Function
>> ** CombineFn
>> ** BinaryCombineFn
>> ** Combine PerKey
>> * Flatten
>> * Partition
>>
>> # Common Transforms
>> * Filter
>> ** ParDo
>> ** Filter
>> * Aggregation
>> ** Count
>> ** Sum
>> ** Avg
>> ** Min / Smallest
>> ** Max / Largest
>>
>> I have plans to add more lessons to make it more complete, covering more
>> Beam fundamentals, e.g. composite transform, branching (multiple
>> transforms, side output), merging (side input), I/O, windowing, streaming,
>> examples (word count, etc), and possibly use case patterns.
>>
>> Please find some screenshots on how it looks like for a student:
>> [image: image.png]
>>
>> [image: image.png]
>>
>> Regards,
>> Henry Suryawirawan
>>
>
Re: Contributing Beam Kata (Java & Python)
Posted by .
Hi Pablo,
The file structure at the moment is structured around the language.
* beam-kata/
* beam-kata/java/
* beam-kata/java/<idea-project>
* beam-kata/python/
* beam-kata/python/<pycharm-project>
Adding a new language in the future should be quite easy.
For students to use the kata, they actually don't need to checkout the repository.
We can create a course archive (zip file) that could be versioned and released on GitHub.
Another alternative is to release it on the Beam website.
The course archive zip can then be opened by the IDE to start the learning.
IntelliJ/PyCharm itself has integration with Stepik (https://stepik.org) which allows an easy way to start the learning.
The course will be downloaded from Stepik to the student's local workspace within seconds.
When I'm available, I will try to create a video to show how the kata looks like.
On 2019/04/18 20:41:13, Pablo Estrada <pa...@google.com> wrote:
> Hi Henry!
> this seems quite nice! For my information, what does the file structure
> look like for something like this? Also, maybe it's not the intended way of
> introducing it, but perhaps, could you share a video showing how one would
> try out the first kata?
>
> Something that comes to mind for having this in Beam is having a
> `learning/katas/` subdirectory in the repository, with a README; and pehaps
> a section on the website for people to try this out.
>
> Another question that comes to mind: Is it possible to structure it so that
> katas for other languages can be added?
>
> Thoughts from others?
> Best
> -P.
Re: Contributing Beam Kata (Java & Python)
Posted by Pablo Estrada <pa...@google.com>.
Hi Henry!
this seems quite nice! For my information, what does the file structure
look like for something like this? Also, maybe it's not the intended way of
introducing it, but perhaps, could you share a video showing how one would
try out the first kata?
Something that comes to mind for having this in Beam is having a
`learning/katas/` subdirectory in the repository, with a README; and pehaps
a section on the website for people to try this out.
Another question that comes to mind: Is it possible to structure it so that
katas for other languages can be added?
Thoughts from others?
Best
-P.
On Thu, Apr 18, 2019 at 1:20 PM Henry Suryawirawan <hs...@google.com>
wrote:
> Hi Beam Devs,
>
> I have been working on what I call Beam Kata, a set of interactive Beam
> coding exercises (i.e. code kata) that aids in learning Beam programming
> hands-on. It is built based on JetBrains Educational Products (
> https://www.jetbrains.com/education/).
> The objective of Beam Kata is to provide a structured hands-on learning
> experience for people to learn about Beam and its SDK by solving focused
> mini problems with gradually increasing complexity.
> I have used the Kata to conduct Beam workshops, teaching the participants
> on how to code in Beam, and so far the feedback has been great.
> They particularly like the hands-on and how it is closely integrated with
> the IDE, giving the best developer experience, especially for
> IntelliJ/PyCharm developers.
>
> I would like to contribute this to the Beam repository and would like to
> check with you whether this is something that would be useful as an
> addition.
> If so, can you advise me in which directory I should commit my work?
> If you need further information, please do let me know.
>
> As of the moment, I have created lessons for both Java & Python SDK.
> The current lessons are:
> # Introduction
> * Hello Beam
>
> # Core Transforms
> * Map
> ** ParDo
> ** ParDo OneToMany
> ** MapElements / Map
> ** FlatMapElements / FlatMap
> * GroupByKey
> * CoGroupByKey
> * Combine
> ** Simple Function
> ** CombineFn
> ** BinaryCombineFn
> ** Combine PerKey
> * Flatten
> * Partition
>
> # Common Transforms
> * Filter
> ** ParDo
> ** Filter
> * Aggregation
> ** Count
> ** Sum
> ** Avg
> ** Min / Smallest
> ** Max / Largest
>
> I have plans to add more lessons to make it more complete, covering more
> Beam fundamentals, e.g. composite transform, branching (multiple
> transforms, side output), merging (side input), I/O, windowing, streaming,
> examples (word count, etc), and possibly use case patterns.
>
> Please find some screenshots on how it looks like for a student:
> [image: image.png]
>
> [image: image.png]
>
> Regards,
> Henry Suryawirawan
>