You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@beam.apache.org by Rose Nguyen <rt...@google.com> on 2019/03/04 17:16:32 UTC
Apache Beam Newsletter - February/March 2019
[image: Beam.png]
February-March 2019 | Newsletter
What’s been done
------------------------------
Apache Beam 2.10.0 released (by: many contributors)
-
Download the release here.
<https://beam.apache.org/get-started/downloads/>
-
See the blog post
<https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html> for more
details.
Apache Beam awarded the 2019 Technology of the Year Award!
-
InfoWorld just awarded Beam the 2019 Technology of the Year Award.
-
See this article
<https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html?nsdr=true>
for more details.
Kettle Beam 0.5 released with support for flink (by: Matt Casters)
-
Kettle now supports Apache Flink as well as Cloud Dataflow and Spark.
-
See Matt’s Blog
<http://sandbox.kettle.be/wordpress/index.php/2019/02/24/kettle-beam-update-0-5-0/>
for more details.
What we’re working on...
------------------------------
Apache Beam 2.11.0 release (by: many contributors)
Hive Metastore Table provider for SQL (by: Anton Kedin)
-
Support for plugging table providers through Beam SQL API to allow
obtaining table schemas from external sources.
-
See the PR <https://github.com/apache/beam/pull/7746> for more details.
User Defined Coders for the Beam Go SDK (by: Robert Burke)
-
Working on expanding the variety of user defined types that can be a
member of a PCollection in the Go SDK.
-
See BEAM-3306 <https://issues.apache.org/jira/browse/BEAM-3306> for more
details.
Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen, Mark Liu, Robbe
Sneyders, Juta Staes, Valentyn Tymofieiev)
-
Beam 2.11.0 is the first release offering partial Python 3 support.
-
Many thanks to all contributors who helped to reach this milestone.
-
IO availablility on Python 3 is currently limited and only Python 3.5
version has been tested extensively.
-
Stay tuned on BEAM-1251 for more details.
Notebooks for quickstarts and custom I/O (by: David Cavazos)
-
Adding IPython notebooks and snippets
-
See [BEAM-6557] <https://github.com/apache/beam/pull/7679> for more
details.
New members
------------------------------
New PMC member!
-
Etienne Chauchot, Nantes, France
New Committers!
-
Gleb Kanterov, Stockholm, Sweden
-
Michael Luckey
New Contributors!
-
Kyle Weaver, San Francisco, CA
-
Would like to help begin implementing portability support for the
Spark runner
-
Tanay Tummapalli, Delhi, India
-
Would like to contribute to Open Source this summer as part of Google
Summer of Code
-
Brian Hulette, Seattle, WA
-
Contributing to Beam Portability
-
Michał Walenia, Warsaw, Poland
-
Working on integration and load testing
-
Daniel Chen, San Francisco, CA
-
Working on Beam Samza runner
Talks & meetups
------------------------------
Plugin Machine Intelligence and Apache Beam with Pentaho - Feb 7 @ London
-
Watch the How to Run Kettle on Apache Beam video here
<https://skillsmatter.com/skillscasts/13405-how-to-run-kettle-on-apache-beam#video>.
-
See event details here
<https://www.meetup.com/Pentaho-London-User-Group/events/256773962/>..
Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @ San Francisco, CA
-
Organized by Thomas Weise and Austin Bennet, with speakers Tyler Akidau,
Robert Crowe, Thomas Weise and Amar Pai
-
See event details here
<https://www.meetup.com/San-Francisco-Apache-Beam/events/257482350/> and
the slides for these presentation: Overview of Apache Beam and
TensorFlow Transform (TFX) with Apache Beam
<http://s.apache.org/beam-intro-feb-2019>, Python Streaming Pipelines
with Beam on Flink
<http://go.lyft.com/python-flink-beam-meetup-2019>, Dynamic
pricing of Lyft rides using streaming
<https://www.slideshare.net/AmarPai2/dynamic-pricing-of-lyft-rides-using-streaming>
.
Flink meetup - Feb 21@ Seattle, WA
-
Speakers from Alibaba, Google, and Uber gave talks about Apache Flink
with Hive, Tensorflow, Beam, and AthenaX.
-
See event details here
<https://www.meetup.com/seattle-flink/events/258723322/> and
presentations here <https://www.slideshare.net/BowenLi9/presentations>.
Beam Summit Europe 2019 - June 19-20 @ Berlin
-
Beam Summit Europe 2019 will take place in Berlin on June 19-20.
-
Speaker CfP and other details to follow soon!
-
Twitter announcement!
<https://twitter.com/matthiasbaetens/status/1098854758893273088>
Resources
------------------------------
Apache Jira Beginner’s Guide (by: Daniel Oliveira)
-
A guide
<https://cwiki.apache.org/confluence/display/BEAM/Beam+Jira+Beginner%27s+Guide>
to introduce Beam contributors to the basics of using the Apache Jira for
Beam development. Feedback welcomed!
An approach to community building from Apache Beam (by: Kenn Knowles)
-
The Apache Software Foundation has published committer guidelines to
help Beam's community building work.
-
See the post <https://blogs.apache.org/comdev/date/20190222> on the ASF
blog.
Exploring Beam SQL on Google Cloud Platform (by: Graham Polley)
-
“In this article, I’ll dive into this new feature of Beam, and see how
it works by using a pipeline to read a data file from GCS, transform it,
and then perform a basic calculation on the values contained in the file”.
-
See article
<https://medium.com/weareservian/exploring-beam-sql-on-google-cloud-platform-b6c77f9b4af4>
and full source code
<https://github.com/polleyg/gcp-batch-ingestion-bigquery/blob/beam_sql/src/main/java/org/polleyg/BeamSQLMagic.java>
.
*Until Next Time!*
--
Rose Thị Nguyễn
Re: Apache Beam Newsletter - February/March 2019
Posted by Rose Nguyen <rt...@google.com>.
Suneel--I responded with some options
Dan--Yes! Please do share! 😀 Here is the doc link:
https://docs.google.com/document/d/1fgG11CVsH31Jflws8ON6DcAr5yEopBT30srdaQk2nQA
On Mon, Mar 4, 2019 at 10:25 AM Dan <da...@dankeeley.co.uk> wrote:
> This is excellent. Is there a link to share this?
>
> Sent from my phone
>
> On Mon, 4 Mar 2019, 5:16 pm Rose Nguyen, <rt...@google.com> wrote:
>
>>
>> [image: Beam.png]
>>
>> February-March 2019 | Newsletter
>>
>> What’s been done
>>
>> ------------------------------
>>
>> Apache Beam 2.10.0 released (by: many contributors)
>>
>> -
>>
>> Download the release here.
>> <https://beam.apache.org/get-started/downloads/>
>> -
>>
>> See the blog post
>> <https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html> for more
>> details.
>>
>>
>> Apache Beam awarded the 2019 Technology of the Year Award!
>>
>> -
>>
>> InfoWorld just awarded Beam the 2019 Technology of the Year Award.
>> -
>>
>> See this article
>> <https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html?nsdr=true>
>> for more details.
>>
>>
>> Kettle Beam 0.5 released with support for flink (by: Matt Casters)
>>
>> -
>>
>> Kettle now supports Apache Flink as well as Cloud Dataflow and Spark.
>> -
>>
>> See Matt’s Blog
>> <http://sandbox.kettle.be/wordpress/index.php/2019/02/24/kettle-beam-update-0-5-0/>
>> for more details.
>>
>>
>>
>> What we’re working on...
>>
>> ------------------------------
>>
>> Apache Beam 2.11.0 release (by: many contributors)
>>
>>
>> Hive Metastore Table provider for SQL (by: Anton Kedin)
>>
>> -
>>
>> Support for plugging table providers through Beam SQL API to allow
>> obtaining table schemas from external sources.
>> -
>>
>> See the PR <https://github.com/apache/beam/pull/7746> for more
>> details.
>>
>>
>> User Defined Coders for the Beam Go SDK (by: Robert Burke)
>>
>> -
>>
>> Working on expanding the variety of user defined types that can be a
>> member of a PCollection in the Go SDK.
>> -
>>
>> See BEAM-3306 <https://issues.apache.org/jira/browse/BEAM-3306> for
>> more details.
>>
>>
>> Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen, Mark Liu,
>> Robbe Sneyders, Juta Staes, Valentyn Tymofieiev)
>>
>> -
>>
>> Beam 2.11.0 is the first release offering partial Python 3 support.
>> -
>>
>> Many thanks to all contributors who helped to reach this milestone.
>> -
>>
>> IO availablility on Python 3 is currently limited and only Python 3.5
>> version has been tested extensively.
>> -
>>
>> Stay tuned on BEAM-1251 for more details.
>>
>>
>> Notebooks for quickstarts and custom I/O (by: David Cavazos)
>>
>> -
>>
>> Adding IPython notebooks and snippets
>> -
>>
>> See [BEAM-6557] <https://github.com/apache/beam/pull/7679> for more
>> details.
>>
>>
>>
>>
>> New members
>> ------------------------------
>>
>> New PMC member!
>>
>> -
>>
>> Etienne Chauchot, Nantes, France
>>
>>
>> New Committers!
>>
>> -
>>
>> Gleb Kanterov, Stockholm, Sweden
>> -
>>
>> Michael Luckey
>>
>>
>> New Contributors!
>>
>> -
>>
>> Kyle Weaver, San Francisco, CA
>> -
>>
>> Would like to help begin implementing portability support for the
>> Spark runner
>> -
>>
>> Tanay Tummapalli, Delhi, India
>> -
>>
>> Would like to contribute to Open Source this summer as part of
>> Google Summer of Code
>> -
>>
>> Brian Hulette, Seattle, WA
>> -
>>
>> Contributing to Beam Portability
>> -
>>
>> Michał Walenia, Warsaw, Poland
>> -
>>
>> Working on integration and load testing
>> -
>>
>> Daniel Chen, San Francisco, CA
>> -
>>
>> Working on Beam Samza runner
>>
>>
>>
>> Talks & meetups
>> ------------------------------
>>
>>
>> Plugin Machine Intelligence and Apache Beam with Pentaho - Feb 7 @ London
>>
>> -
>>
>> Watch the How to Run Kettle on Apache Beam video here
>> <https://skillsmatter.com/skillscasts/13405-how-to-run-kettle-on-apache-beam#video>.
>>
>> -
>>
>> See event details here
>> <https://www.meetup.com/Pentaho-London-User-Group/events/256773962/>..
>>
>>
>> Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @ San Francisco,
>> CA
>>
>> -
>>
>> Organized by Thomas Weise and Austin Bennet, with speakers Tyler
>> Akidau, Robert Crowe, Thomas Weise and Amar Pai
>> -
>>
>> See event details here
>> <https://www.meetup.com/San-Francisco-Apache-Beam/events/257482350/>
>> and the slides for these presentation: Overview of Apache Beam and
>> TensorFlow Transform (TFX) with Apache Beam
>> <http://s.apache.org/beam-intro-feb-2019>, Python Streaming Pipelines
>> with Beam on Flink <http://go.lyft.com/python-flink-beam-meetup-2019>,
>> Dynamic pricing of Lyft rides using streaming
>> <https://www.slideshare.net/AmarPai2/dynamic-pricing-of-lyft-rides-using-streaming>
>>
>> .
>>
>> Flink meetup - Feb 21@ Seattle, WA
>>
>> -
>>
>> Speakers from Alibaba, Google, and Uber gave talks about Apache Flink
>> with Hive, Tensorflow, Beam, and AthenaX.
>> -
>>
>> See event details here
>> <https://www.meetup.com/seattle-flink/events/258723322/> and
>> presentations here <https://www.slideshare.net/BowenLi9/presentations>.
>>
>>
>>
>> Beam Summit Europe 2019 - June 19-20 @ Berlin
>>
>> -
>>
>> Beam Summit Europe 2019 will take place in Berlin on June 19-20.
>> -
>>
>> Speaker CfP and other details to follow soon!
>> -
>>
>> Twitter announcement!
>> <https://twitter.com/matthiasbaetens/status/1098854758893273088>
>>
>>
>>
>> Resources
>> ------------------------------
>>
>> Apache Jira Beginner’s Guide (by: Daniel Oliveira)
>>
>> -
>>
>> A guide
>> <https://cwiki.apache.org/confluence/display/BEAM/Beam+Jira+Beginner%27s+Guide>
>> to introduce Beam contributors to the basics of using the Apache Jira for
>> Beam development. Feedback welcomed!
>>
>>
>> An approach to community building from Apache Beam (by: Kenn Knowles)
>>
>> -
>>
>> The Apache Software Foundation has published committer guidelines to
>> help Beam's community building work.
>> -
>>
>> See the post <https://blogs.apache.org/comdev/date/20190222> on the
>> ASF blog.
>>
>>
>> Exploring Beam SQL on Google Cloud Platform (by: Graham Polley)
>>
>> -
>>
>> “In this article, I’ll dive into this new feature of Beam, and see
>> how it works by using a pipeline to read a data file from GCS, transform
>> it, and then perform a basic calculation on the values contained in the
>> file”.
>> -
>>
>> See article
>> <https://medium.com/weareservian/exploring-beam-sql-on-google-cloud-platform-b6c77f9b4af4>
>> and full source code
>> <https://github.com/polleyg/gcp-batch-ingestion-bigquery/blob/beam_sql/src/main/java/org/polleyg/BeamSQLMagic.java>
>> .
>>
>>
>> *Until Next Time!*
>> --
>> Rose Thị Nguyễn
>>
>
--
Rose Thị Nguyễn
Re: Apache Beam Newsletter - February/March 2019
Posted by Dan <da...@dankeeley.co.uk>.
This is excellent. Is there a link to share this?
Sent from my phone
On Mon, 4 Mar 2019, 5:16 pm Rose Nguyen, <rt...@google.com> wrote:
>
> [image: Beam.png]
>
> February-March 2019 | Newsletter
>
> What’s been done
>
> ------------------------------
>
> Apache Beam 2.10.0 released (by: many contributors)
>
> -
>
> Download the release here.
> <https://beam.apache.org/get-started/downloads/>
> -
>
> See the blog post
> <https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html> for more
> details.
>
>
> Apache Beam awarded the 2019 Technology of the Year Award!
>
> -
>
> InfoWorld just awarded Beam the 2019 Technology of the Year Award.
> -
>
> See this article
> <https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html?nsdr=true>
> for more details.
>
>
> Kettle Beam 0.5 released with support for flink (by: Matt Casters)
>
> -
>
> Kettle now supports Apache Flink as well as Cloud Dataflow and Spark.
> -
>
> See Matt’s Blog
> <http://sandbox.kettle.be/wordpress/index.php/2019/02/24/kettle-beam-update-0-5-0/>
> for more details.
>
>
>
> What we’re working on...
>
> ------------------------------
>
> Apache Beam 2.11.0 release (by: many contributors)
>
>
> Hive Metastore Table provider for SQL (by: Anton Kedin)
>
> -
>
> Support for plugging table providers through Beam SQL API to allow
> obtaining table schemas from external sources.
> -
>
> See the PR <https://github.com/apache/beam/pull/7746> for more details.
>
>
> User Defined Coders for the Beam Go SDK (by: Robert Burke)
>
> -
>
> Working on expanding the variety of user defined types that can be a
> member of a PCollection in the Go SDK.
> -
>
> See BEAM-3306 <https://issues.apache.org/jira/browse/BEAM-3306> for
> more details.
>
>
> Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen, Mark Liu, Robbe
> Sneyders, Juta Staes, Valentyn Tymofieiev)
>
> -
>
> Beam 2.11.0 is the first release offering partial Python 3 support.
> -
>
> Many thanks to all contributors who helped to reach this milestone.
> -
>
> IO availablility on Python 3 is currently limited and only Python 3.5
> version has been tested extensively.
> -
>
> Stay tuned on BEAM-1251 for more details.
>
>
> Notebooks for quickstarts and custom I/O (by: David Cavazos)
>
> -
>
> Adding IPython notebooks and snippets
> -
>
> See [BEAM-6557] <https://github.com/apache/beam/pull/7679> for more
> details.
>
>
>
>
> New members
> ------------------------------
>
> New PMC member!
>
> -
>
> Etienne Chauchot, Nantes, France
>
>
> New Committers!
>
> -
>
> Gleb Kanterov, Stockholm, Sweden
> -
>
> Michael Luckey
>
>
> New Contributors!
>
> -
>
> Kyle Weaver, San Francisco, CA
> -
>
> Would like to help begin implementing portability support for the
> Spark runner
> -
>
> Tanay Tummapalli, Delhi, India
> -
>
> Would like to contribute to Open Source this summer as part of
> Google Summer of Code
> -
>
> Brian Hulette, Seattle, WA
> -
>
> Contributing to Beam Portability
> -
>
> Michał Walenia, Warsaw, Poland
> -
>
> Working on integration and load testing
> -
>
> Daniel Chen, San Francisco, CA
> -
>
> Working on Beam Samza runner
>
>
>
> Talks & meetups
> ------------------------------
>
>
> Plugin Machine Intelligence and Apache Beam with Pentaho - Feb 7 @ London
>
> -
>
> Watch the How to Run Kettle on Apache Beam video here
> <https://skillsmatter.com/skillscasts/13405-how-to-run-kettle-on-apache-beam#video>.
>
> -
>
> See event details here
> <https://www.meetup.com/Pentaho-London-User-Group/events/256773962/>..
>
>
> Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @ San Francisco,
> CA
>
> -
>
> Organized by Thomas Weise and Austin Bennet, with speakers Tyler
> Akidau, Robert Crowe, Thomas Weise and Amar Pai
> -
>
> See event details here
> <https://www.meetup.com/San-Francisco-Apache-Beam/events/257482350/>
> and the slides for these presentation: Overview of Apache Beam and
> TensorFlow Transform (TFX) with Apache Beam
> <http://s.apache.org/beam-intro-feb-2019>, Python Streaming Pipelines
> with Beam on Flink <http://go.lyft.com/python-flink-beam-meetup-2019>, Dynamic
> pricing of Lyft rides using streaming
> <https://www.slideshare.net/AmarPai2/dynamic-pricing-of-lyft-rides-using-streaming>
>
> .
>
> Flink meetup - Feb 21@ Seattle, WA
>
> -
>
> Speakers from Alibaba, Google, and Uber gave talks about Apache Flink
> with Hive, Tensorflow, Beam, and AthenaX.
> -
>
> See event details here
> <https://www.meetup.com/seattle-flink/events/258723322/> and
> presentations here <https://www.slideshare.net/BowenLi9/presentations>.
>
>
>
> Beam Summit Europe 2019 - June 19-20 @ Berlin
>
> -
>
> Beam Summit Europe 2019 will take place in Berlin on June 19-20.
> -
>
> Speaker CfP and other details to follow soon!
> -
>
> Twitter announcement!
> <https://twitter.com/matthiasbaetens/status/1098854758893273088>
>
>
>
> Resources
> ------------------------------
>
> Apache Jira Beginner’s Guide (by: Daniel Oliveira)
>
> -
>
> A guide
> <https://cwiki.apache.org/confluence/display/BEAM/Beam+Jira+Beginner%27s+Guide>
> to introduce Beam contributors to the basics of using the Apache Jira for
> Beam development. Feedback welcomed!
>
>
> An approach to community building from Apache Beam (by: Kenn Knowles)
>
> -
>
> The Apache Software Foundation has published committer guidelines to
> help Beam's community building work.
> -
>
> See the post <https://blogs.apache.org/comdev/date/20190222> on the
> ASF blog.
>
>
> Exploring Beam SQL on Google Cloud Platform (by: Graham Polley)
>
> -
>
> “In this article, I’ll dive into this new feature of Beam, and see how
> it works by using a pipeline to read a data file from GCS, transform it,
> and then perform a basic calculation on the values contained in the file”.
> -
>
> See article
> <https://medium.com/weareservian/exploring-beam-sql-on-google-cloud-platform-b6c77f9b4af4>
> and full source code
> <https://github.com/polleyg/gcp-batch-ingestion-bigquery/blob/beam_sql/src/main/java/org/polleyg/BeamSQLMagic.java>
> .
>
>
> *Until Next Time!*
> --
> Rose Thị Nguyễn
>
Re: Apache Beam Newsletter - February/March 2019
Posted by Matthias Baetens <ba...@gmail.com>.
Adding my 2 cents:
+1 for adding News to the repository.
I agree with Alexey in that News is different from blogpost (tend to be
bigger and might have technical content and/or debriefs from events).
Tying newsletter to releases also don't fully make sense from an event
point of view - these don't tend to be scheduled 3 months in advance unless
they are very big. A cadence of once every other month like Pablo suggested
makes sense from that point of view.
On Wed, 13 Mar 2019 at 17:02, Ahmet Altay <al...@google.com> wrote:
> Related to release notes, I do agree with cleaninp up the JIRA (as a
> community) and including the dependency changes. I find those information
> useful.
>
> If the proposal here is to loosely align newsletter timelines with release
> times, I agree with that. If it is a proposal to merge release notes with
> the newsletter, I will be worried it will make the release process more
> difficult than it is today. Simply because we will need more things to
> happen for a release to be completed.
>
> On Wed, Mar 13, 2019 at 2:37 AM Etienne Chauchot <ec...@apache.org>
> wrote:
>
>> Hi,
>> +1 on the process
>> +1 on the new News section
>>
>> I also think that matching the newsletter pace with the release pace
>> makes it clearer for users. Also, as a general obvious comment, newsletter
>> have the interest of providing the ongoing work compared to release notes
>> that provide only the work that was done.
>>
>> Etienne
>>
>> Le mardi 12 mars 2019 à 21:50 -0700, Kenneth Knowles a écrit :
>>
>>
>>
>> On Tue, Mar 12, 2019 at 9:04 PM Thomas Weise <th...@apache.org> wrote:
>>
>> The release blog is already on the radar for improvement [1].
>>
>> I don't think there is a need to separate out the release blogs. That's
>> if they provide content that is valuable to users (IMO that's not exactly
>> the case today).
>>
>>
>> The case I am thinking about is a user searching the web for an issue and
>> figuring out what version it was fixed in. Dep upgrades being a primary
>> thing to notice.
>>
>>
>> If release blogs are just reformatted JIRA reports, then maybe we can
>> skip them altogether (release notes are already linked from download page).
>>
>>
>> One reason is that Jira remains mutable and I am not so sure of the SEO,
>> in terms of helping users figure out if an upgrade might solve their
>> problem.
>>
>>
>> In that case I would much rather like to see the original JIRA report
>> cleaned up as part of the release process.
>>
>>
>> Either way, this seems worthwhile. I think release managers often do this
>> anyhow. How could we formalize it? Do we need to? My favorite form of
>> formal process is just to get more eyes on some LGTM.
>>
>>
>> On the other hand, if release blogs are assembled similar to the
>> newsletter that we discuss here, through broader contributor input and with
>> the aim to provide more meaningful content to users, then I don't see why
>> we need to put them into a different area.
>>
>>
>> Same answer: people searching for issue resolution. I wonder if our
>> release notes should be a static copy/paste of the Jira list, deleting
>> things that really make no sense and editing everything else to be
>> meaningful. But I get the idea that we don't really need a separate area.
>>
>> What if newsletters matched releases, and could be drafted throughout the
>> 6 week period, with folks really trying to give a narrative to what they
>> contributed to a release?
>>
>> Kenn
>>
>>
>> Overall, given proposed newsletter cadence of 2 months and existing
>> release cadence, we would probably still end up with a monthly piece of
>> news.
>>
>> Thanks,
>> Thomas
>>
>>
>>
>> https://lists.apache.org/thread.html/224908775d8807eecc53e45f14f99d6a439beb8bb5165e3a813bf82d@%3Cdev.beam.apache.org%3E
>>
>>
>> On Tue, Mar 12, 2019 at 9:27 AM Rose Nguyen <rt...@google.com> wrote:
>>
>> Once every 2 months works. We include both big and small items. It'll
>> provide the focus Thomas mentions but still catches the frequency that
>> Pablo suggested for relevancy. To Alexey's point, it's difficult to
>> navigate the more recent non-release blog posts (<1 year) because they are
>> sprinkled in.
>>
>> A compromise for all of our points is to move the release notes to a
>> separate section, and have a single section that's blog/news.
>>
>> And thanks for wanting to contribute, Aizhamal! Teaming up with you will
>> make things run smoothly.
>>
>> On Tue, Mar 12, 2019 at 3:58 AM Alexey Romanenko <
>> aromanenko.dev@gmail.com> wrote:
>>
>> +1 for “News” section. Though, I’d propose to exclude “release notes”
>> posts from “Blog” section list (since now we have quite regular releases,
>> it makes blog posts list not very readable for users) and move them to new
>> created News section. Also, this section could include the posts about
>> other Beam events, like meet-ups, conferences, project updates, etc. I’d
>> keep “Blog” section for more technical posts.
>>
>> On 12 Mar 2019, at 06:31, Pablo Estrada <pa...@google.com> wrote:
>>
>> I agree that the newsletter fits well as a blog post. I think that'd work
>> best.
>>
>> As for the cadence, I think quarterly would be a bit too infrequent. I
>> like once a month, or once every other month to have at least one per
>> release. Though happy to hear other people's thoughts.
>> Best
>> -P.
>>
>> On Mon, Mar 11, 2019 at 6:42 PM Thomas Weise <th...@apache.org> wrote:
>>
>> +1 for single blog/news section
>>
>> Also I wouldn't mind quarterly cadence, to provide more focus for folks
>> to contribute.
>>
>> On Mon, Mar 11, 2019 at 6:35 PM Kenneth Knowles <ke...@apache.org> wrote:
>>
>> Could the newsletter be a blog entry? If you check out
>> https://blogs.apache.org/ many of the posts are "Apache News Round-up".
>> We could rename the "Blog" section to "News" if you ask me.
>>
>> Kenn
>>
>> On Mon, Mar 11, 2019 at 5:13 PM Aizhamal Nurmamat kyzy <
>> aizhamal@google.com> wrote:
>>
>> Hello everyone,
>>
>> I had a chat with Rose on how I can support the effort and keep sending
>> the newsletters on a monthly basis. The new workflow would look as follows:
>>
>> 1. Send out the same [CALL FOR ITEMS] where you can contribute to the
>> Google doc with deadlines
>> 2. Edit the doc after the deadline
>> 3. Convert the file into Markdown
>> 4. Send in PR to add the file to Beam repo in Newsletter directory
>> 5. Have people send their fixes/updates through PRs
>>
>> In this effort, I can support Rose in steps 3 & 4.
>>
>> We would also need:
>>
>> - Create a Newsletter section under the Community tab
>> - Write guidelines on newsletter contributions
>> - Make a note about timing e.g. if upcoming event, then add to the
>> next newsletter
>>
>> How does that sound to you all?
>>
>>
>> On Wed, Mar 6, 2019 at 9:19 PM Rose Nguyen <rt...@google.com> wrote:
>>
>> Good points. With the suggested workflow I think I can support a
>> quarterly newsletter. I'm also happy to get more involvement from others to
>> do this work and we can see what cadence that allows.
>>
>> On Wed, Mar 6, 2019 at 8:22 PM Kenneth Knowles <ke...@apache.org> wrote:
>>
>> Good points Melissa & Austin.
>>
>> - archive in the repo & on the website
>> - put missed items on the next newsletter, so anyone following sees them
>>
>> Kenn
>>
>> On Wed, Mar 6, 2019 at 3:26 PM Suneel Marthi <sm...@apache.org> wrote:
>>
>> I believe there was also a Beam workshop or working session in Warsaw
>> last week.
>>
>> On Wed, Mar 6, 2019 at 6:20 PM Austin Bennett <
>> whatwouldaustindo@gmail.com> wrote:
>>
>> +1 for archive in our repo.
>>
>> I do follow the newsletter, but am unlikely to go back and look into the
>> past for changes/updates.
>>
>> Would suggest that things that get missed in one newsletter (a concrete
>> example, Suneel's talks not mentioned in the newsletter) would get
>> published in the next iteration, rather than editing the past 'published'
>> newsletter. Put another way, save editing the past for corrections (typos,
>> things being incorrect). Else, I imagine that I'm unlikely to catch a
>> great announcement that warranted being in the newsletter in the first
>> place. This certainly works better with a regular/frequent release
>> cadence, like we arrived at for version releases (then, if something misses
>> one cut, it is not too big a deal, as the next release is coming soon).
>>
>>
>>
>>
>> On Wed, Mar 6, 2019 at 12:50 PM Melissa Pashniak <me...@google.com>
>> wrote:
>>
>>
>> For step #2 (publishing onto the website), I think it would be good to
>> stay consistent with our existing workflows if possible. Rather than using
>> an external tool, what about:
>>
>> After a google doc newsletter draft is ready, convert it into a standard
>> markdown file and put it into our GitHub repo, perhaps in a new newsletter
>> directory in the website community directory [1]. These would be listed for
>> browsing on a Newsletters page as mentioned in step #4. People can then
>> just open a PR to add missing things to the pages later, and the newsletter
>> will be automatically updated on the website through our standard website
>> workflow. It also avoids the potential issue of the source google docs
>> disappearing in the future, as they are stored in a community location.
>>
>> [1] https://github.com/apache/beam/tree/master/website/src/community
>>
>>
>> On Wed, Mar 6, 2019 at 10:36 AM Rose Nguyen <rt...@google.com> wrote:
>>
>> I think that would be a great idea to change formats to help with
>> distribution. I'm open to suggestions! I'm currently using a Google doc to
>> collect and edit, then copy/paste sending the newsletter out directly, based
>> on an interpretation of this discussion
>> <https://lists.apache.org/thread.html/1f638eae43fe8abcb2f8752141c96d3dbdac86a583e0790044eea727@%3Cdev.beam.apache.org%3E>
>> .
>>
>> How about this doc->website->Beam site workflow?:
>>
>> 1. The same usual newsletter [CALL FOR ITEMS] where you can
>> contribute to the google doc, with soft deadlines for when I'll publish.
>> 2. I'll publish the doc itself onto a website.
>> 3. The newsletter is mailed out in the same way, but now with a
>> shareable website link.
>> 4. We'll keep an index of archived newsletter web pages on the Beam
>> site, under the Community tab.
>> 5. If you want to submit more content after the soft deadline, add it
>> to the google doc and let me know to republish. I don't want to make the
>> publication changes automatic because that leaves us open to tampering.
>>
>>
>> This process is more laggy, so I'd suggest doing a 2 month vs monthly
>> newsletter cadence. If we're happy with this idea, I'll send in a website
>> PR for a new "Newsletter" left nav item under Community.
>>
>> Here's an example of a published newsletter: Apache Beam February-March
>> 2019
>> <https://gdoc.pub/doc/e/2PACX-1vTQIS4WkxV-HpgX5Lb6q05g4-wuIVcYd82123Mp4Y6q9fMv6Ynwd-l7dI4TrMyCrKilyU-YsoitbnZB>
>>
>>
>> - This link is permanent unless the principal google doc is deleted.
>> - Changes to the google doc after web publication are not
>> automatically published on the website to protect the information integrity.
>> - Republishing is quick and easy for me if you let me know you've
>> added more.
>> - I'll improve the formatting later if we go with this route.
>>
>> Any thoughts?
>>
>> On Wed, Mar 6, 2019 at 6:13 AM Thomas Weise <th...@apache.org> wrote:
>>
>> Similar to blog posts. A link that can be shared would also help to
>> distribute over other channels, such as Twitter.
>>
>>
>> On Wed, Mar 6, 2019, 6:06 AM Ismaël Mejía <ie...@gmail.com> wrote:
>>
>> We should have these newsletters published somewhere with a fixed URL so
>> we can add missing updates, I have at least missed putting stuff in the
>> last 3 ones just to realize when the newsletter has been already
>> 'published' (and sadly interesting features like zstd have not been
>> announced because of this).
>>
>> On Wed, Mar 6, 2019 at 2:48 PM Etienne Chauchot <ec...@apache.org>
>> wrote:
>>
>> Hi,
>> I would add in what's been done:
>>
>> Work on cassandraIO (Etienne Chauchot, Mathieu Blanchard, Frank Shahar) :
>> refactorings, bugfixes, new where clause, security fix
>>
>> Etienne
>>
>> Le lundi 04 mars 2019 à 18:36 +0100, Suneel Marthi a écrit :
>>
>> Is this the final draft? - we had 2 beam talks at Big Data Tech Warsaw
>> last Wednesday - I can send the updates offline.
>>
>> On Mon, Mar 4, 2019 at 6:16 PM Rose Nguyen <rt...@google.com> wrote:
>>
>>
>> [image: Beam.png]
>> February-March 2019 | Newsletter
>>
>> What’s been done
>>
>> ------------------------------
>>
>> Apache Beam 2.10.0 released (by: many contributors)
>>
>> - Download the release here.
>> <https://beam.apache.org/get-started/downloads/>
>> - See the blog post
>> <https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html> for more
>> details.
>>
>>
>> Apache Beam awarded the 2019 Technology of the Year Award!
>>
>> - InfoWorld just awarded Beam the 2019 Technology of the Year Award.
>> - See this article
>> <https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html?nsdr=true>
>> for more details.
>>
>>
>> Kettle Beam 0.5 released with support for flink (by: Matt Casters)
>>
>> - Kettle now supports Apache Flink as well as Cloud Dataflow and
>> Spark.
>> - See Matt’s Blog
>> <http://sandbox.kettle.be/wordpress/index.php/2019/02/24/kettle-beam-update-0-5-0/>
>> for more details.
>>
>>
>>
>> What we’re working on...
>>
>> ------------------------------
>>
>> Apache Beam 2.11.0 release (by: many contributors)
>>
>>
>> Hive Metastore Table provider for SQL (by: Anton Kedin)
>>
>> - Support for plugging table providers through Beam SQL API to allow
>> obtaining table schemas from external sources.
>> - See the PR <https://github.com/apache/beam/pull/7746> for more
>> details.
>>
>>
>> User Defined Coders for the Beam Go SDK (by: Robert Burke)
>>
>> - Working on expanding the variety of user defined types that can be
>> a member of a PCollection in the Go SDK.
>> - See BEAM-3306 <https://issues.apache.org/jira/browse/BEAM-3306> for
>> more details.
>>
>>
>> Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen, Mark Liu,
>> Robbe Sneyders, Juta Staes, Valentyn Tymofieiev)
>>
>> - Beam 2.11.0 is the first release offering partial Python 3 support.
>> - Many thanks to all contributors who helped to reach this milestone.
>> - IO availablility on Python 3 is currently limited and only Python
>> 3.5 version has been tested extensively.
>> - Stay tuned on BEAM-1251 for more details.
>>
>>
>> Notebooks for quickstarts and custom I/O (by: David Cavazos)
>>
>> - Adding IPython notebooks and snippets
>> - See [BEAM-6557] <https://github.com/apache/beam/pull/7679> for more
>> details.
>>
>>
>>
>>
>> New members
>> ------------------------------
>>
>> New PMC member!
>>
>> - Etienne Chauchot, Nantes, France
>>
>>
>> New Committers!
>>
>> - Gleb Kanterov, Stockholm, Sweden
>> - Michael Luckey
>>
>>
>> New Contributors!
>>
>> - Kyle Weaver, San Francisco, CA
>> - Would like to help begin implementing portability support for the
>> Spark runner
>> - Tanay Tummapalli, Delhi, India
>> - Would like to contribute to Open Source this summer as part of
>> Google Summer of Code
>> - Brian Hulette, Seattle, WA
>> - Contributing to Beam Portability
>> - Michał Walenia, Warsaw, Poland
>> - Working on integration and load testing
>> - Daniel Chen, San Francisco, CA
>> - Working on Beam Samza runner
>>
>>
>>
>> Talks & meetups
>> ------------------------------
>>
>>
>> Plugin Machine Intelligence and Apache Beam with Pentaho - Feb 7 @ London
>>
>> - Watch the How to Run Kettle on Apache Beam video here
>> <https://skillsmatter.com/skillscasts/13405-how-to-run-kettle-on-apache-beam#video>.
>>
>> - See event details here
>> <https://www.meetup.com/Pentaho-London-User-Group/events/256773962/>..
>>
>>
>> Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @ San Francisco,
>> CA
>>
>> - Organized by Thomas Weise and Austin Bennet, with speakers Tyler
>> Akidau, Robert Crowe, Thomas Weise and Amar Pai
>> - See event details here
>> <https://www.meetup.com/San-Francisco-Apache-Beam/events/257482350/>
>> and the slides for these presentation: Overview of Apache Beam and
>> TensorFlow Transform (TFX) with Apache Beam
>> <http://s.apache.org/beam-intro-feb-2019>, Python Streaming Pipelines
>> with Beam on Flink <http://go.lyft.com/python-flink-beam-meetup-2019>,
>> Dynamic pricing of Lyft rides using streaming
>> <https://www.slideshare.net/AmarPai2/dynamic-pricing-of-lyft-rides-using-streaming>
>>
>> .
>> Flink meetup - Feb 21@ Seattle, WA
>>
>> - Speakers from Alibaba, Google, and Uber gave talks about Apache
>> Flink with Hive, Tensorflow, Beam, and AthenaX.
>> - See event details here
>> <https://www.meetup.com/seattle-flink/events/258723322/> and
>> presentations here <https://www.slideshare.net/BowenLi9/presentations>.
>>
>>
>>
>> Beam Summit Europe 2019 - June 19-20 @ Berlin
>>
>> - Beam Summit Europe 2019 will take place in Berlin on June 19-20.
>> - Speaker CfP and other details to follow soon!
>> - Twitter announcement!
>> <https://twitter.com/matthiasbaetens/status/1098854758893273088>
>>
>>
>>
>> Resources
>> ------------------------------
>>
>> Apache Jira Beginner’s Guide (by: Daniel Oliveira)
>>
>> - A guide
>> <https://cwiki.apache.org/confluence/display/BEAM/Beam+Jira+Beginner%27s+Guide>
>> to introduce Beam contributors to the basics of using the Apache Jira for
>> Beam development. Feedback welcomed!
>>
>>
>> An approach to community building from Apache Beam (by: Kenn Knowles)
>>
>> - The Apache Software Foundation has published committer guidelines
>> to help Beam's community building work.
>> - See the post <https://blogs.apache.org/comdev/date/20190222> on the
>> ASF blog.
>>
>>
>> Exploring Beam SQL on Google Cloud Platform (by: Graham Polley)
>>
>> - “In this article, I’ll dive into this new feature of Beam, and see
>> how it works by using a pipeline to read a data file from GCS, transform
>> it, and then perform a basic calculation on the values contained in the
>> file”.
>> - See article
>> <https://medium.com/weareservian/exploring-beam-sql-on-google-cloud-platform-b6c77f9b4af4>
>> and full source code
>> <https://github.com/polleyg/gcp-batch-ingestion-bigquery/blob/beam_sql/src/main/java/org/polleyg/BeamSQLMagic.java>
>> .
>>
>>
>> *Until Next Time!*
>> --
>> Rose Thị Nguyễn
>>
>>
>>
>> --
>> Rose Thị Nguyễn
>>
>>
>>
>> --
>> Rose Thị Nguyễn
>>
>>
>>
>> --
>>
>> *Aizhamal Nurmamat kyzy*
>> Open Source Program Manager
>> 646-355-9740 <(646)%20355-9740> Mobile
>> 601 North 34th Street, Seattle, WA 98103
>> <https://maps.google.com/?q=601+North+34th+Street,+Seattle,+WA+98103&entry=gmail&source=g>
>>
>>
>>
>>
>>
Re: Apache Beam Newsletter - February/March 2019
Posted by Ahmet Altay <al...@google.com>.
Related to release notes, I do agree with cleaninp up the JIRA (as a
community) and including the dependency changes. I find those information
useful.
If the proposal here is to loosely align newsletter timelines with release
times, I agree with that. If it is a proposal to merge release notes with
the newsletter, I will be worried it will make the release process more
difficult than it is today. Simply because we will need more things to
happen for a release to be completed.
On Wed, Mar 13, 2019 at 2:37 AM Etienne Chauchot <ec...@apache.org>
wrote:
> Hi,
> +1 on the process
> +1 on the new News section
>
> I also think that matching the newsletter pace with the release pace makes
> it clearer for users. Also, as a general obvious comment, newsletter have
> the interest of providing the ongoing work compared to release notes that
> provide only the work that was done.
>
> Etienne
>
> Le mardi 12 mars 2019 à 21:50 -0700, Kenneth Knowles a écrit :
>
>
>
> On Tue, Mar 12, 2019 at 9:04 PM Thomas Weise <th...@apache.org> wrote:
>
> The release blog is already on the radar for improvement [1].
>
> I don't think there is a need to separate out the release blogs. That's if
> they provide content that is valuable to users (IMO that's not exactly the
> case today).
>
>
> The case I am thinking about is a user searching the web for an issue and
> figuring out what version it was fixed in. Dep upgrades being a primary
> thing to notice.
>
>
> If release blogs are just reformatted JIRA reports, then maybe we can skip
> them altogether (release notes are already linked from download page).
>
>
> One reason is that Jira remains mutable and I am not so sure of the SEO,
> in terms of helping users figure out if an upgrade might solve their
> problem.
>
>
> In that case I would much rather like to see the original JIRA report
> cleaned up as part of the release process.
>
>
> Either way, this seems worthwhile. I think release managers often do this
> anyhow. How could we formalize it? Do we need to? My favorite form of
> formal process is just to get more eyes on some LGTM.
>
>
> On the other hand, if release blogs are assembled similar to the
> newsletter that we discuss here, through broader contributor input and with
> the aim to provide more meaningful content to users, then I don't see why
> we need to put them into a different area.
>
>
> Same answer: people searching for issue resolution. I wonder if our
> release notes should be a static copy/paste of the Jira list, deleting
> things that really make no sense and editing everything else to be
> meaningful. But I get the idea that we don't really need a separate area.
>
> What if newsletters matched releases, and could be drafted throughout the
> 6 week period, with folks really trying to give a narrative to what they
> contributed to a release?
>
> Kenn
>
>
> Overall, given proposed newsletter cadence of 2 months and existing
> release cadence, we would probably still end up with a monthly piece of
> news.
>
> Thanks,
> Thomas
>
>
>
> https://lists.apache.org/thread.html/224908775d8807eecc53e45f14f99d6a439beb8bb5165e3a813bf82d@%3Cdev.beam.apache.org%3E
>
>
> On Tue, Mar 12, 2019 at 9:27 AM Rose Nguyen <rt...@google.com> wrote:
>
> Once every 2 months works. We include both big and small items. It'll
> provide the focus Thomas mentions but still catches the frequency that
> Pablo suggested for relevancy. To Alexey's point, it's difficult to
> navigate the more recent non-release blog posts (<1 year) because they are
> sprinkled in.
>
> A compromise for all of our points is to move the release notes to a
> separate section, and have a single section that's blog/news.
>
> And thanks for wanting to contribute, Aizhamal! Teaming up with you will
> make things run smoothly.
>
> On Tue, Mar 12, 2019 at 3:58 AM Alexey Romanenko <ar...@gmail.com>
> wrote:
>
> +1 for “News” section. Though, I’d propose to exclude “release notes”
> posts from “Blog” section list (since now we have quite regular releases,
> it makes blog posts list not very readable for users) and move them to new
> created News section. Also, this section could include the posts about
> other Beam events, like meet-ups, conferences, project updates, etc. I’d
> keep “Blog” section for more technical posts.
>
> On 12 Mar 2019, at 06:31, Pablo Estrada <pa...@google.com> wrote:
>
> I agree that the newsletter fits well as a blog post. I think that'd work
> best.
>
> As for the cadence, I think quarterly would be a bit too infrequent. I
> like once a month, or once every other month to have at least one per
> release. Though happy to hear other people's thoughts.
> Best
> -P.
>
> On Mon, Mar 11, 2019 at 6:42 PM Thomas Weise <th...@apache.org> wrote:
>
> +1 for single blog/news section
>
> Also I wouldn't mind quarterly cadence, to provide more focus for folks to
> contribute.
>
> On Mon, Mar 11, 2019 at 6:35 PM Kenneth Knowles <ke...@apache.org> wrote:
>
> Could the newsletter be a blog entry? If you check out
> https://blogs.apache.org/ many of the posts are "Apache News Round-up".
> We could rename the "Blog" section to "News" if you ask me.
>
> Kenn
>
> On Mon, Mar 11, 2019 at 5:13 PM Aizhamal Nurmamat kyzy <
> aizhamal@google.com> wrote:
>
> Hello everyone,
>
> I had a chat with Rose on how I can support the effort and keep sending
> the newsletters on a monthly basis. The new workflow would look as follows:
>
> 1. Send out the same [CALL FOR ITEMS] where you can contribute to the
> Google doc with deadlines
> 2. Edit the doc after the deadline
> 3. Convert the file into Markdown
> 4. Send in PR to add the file to Beam repo in Newsletter directory
> 5. Have people send their fixes/updates through PRs
>
> In this effort, I can support Rose in steps 3 & 4.
>
> We would also need:
>
> - Create a Newsletter section under the Community tab
> - Write guidelines on newsletter contributions
> - Make a note about timing e.g. if upcoming event, then add to the
> next newsletter
>
> How does that sound to you all?
>
>
> On Wed, Mar 6, 2019 at 9:19 PM Rose Nguyen <rt...@google.com> wrote:
>
> Good points. With the suggested workflow I think I can support a quarterly
> newsletter. I'm also happy to get more involvement from others to do this
> work and we can see what cadence that allows.
>
> On Wed, Mar 6, 2019 at 8:22 PM Kenneth Knowles <ke...@apache.org> wrote:
>
> Good points Melissa & Austin.
>
> - archive in the repo & on the website
> - put missed items on the next newsletter, so anyone following sees them
>
> Kenn
>
> On Wed, Mar 6, 2019 at 3:26 PM Suneel Marthi <sm...@apache.org> wrote:
>
> I believe there was also a Beam workshop or working session in Warsaw last
> week.
>
> On Wed, Mar 6, 2019 at 6:20 PM Austin Bennett <wh...@gmail.com>
> wrote:
>
> +1 for archive in our repo.
>
> I do follow the newsletter, but am unlikely to go back and look into the
> past for changes/updates.
>
> Would suggest that things that get missed in one newsletter (a concrete
> example, Suneel's talks not mentioned in the newsletter) would get
> published in the next iteration, rather than editing the past 'published'
> newsletter. Put another way, save editing the past for corrections (typos,
> things being incorrect). Else, I imagine that I'm unlikely to catch a
> great announcement that warranted being in the newsletter in the first
> place. This certainly works better with a regular/frequent release
> cadence, like we arrived at for version releases (then, if something misses
> one cut, it is not too big a deal, as the next release is coming soon).
>
>
>
>
> On Wed, Mar 6, 2019 at 12:50 PM Melissa Pashniak <me...@google.com>
> wrote:
>
>
> For step #2 (publishing onto the website), I think it would be good to
> stay consistent with our existing workflows if possible. Rather than using
> an external tool, what about:
>
> After a google doc newsletter draft is ready, convert it into a standard
> markdown file and put it into our GitHub repo, perhaps in a new newsletter
> directory in the website community directory [1]. These would be listed for
> browsing on a Newsletters page as mentioned in step #4. People can then
> just open a PR to add missing things to the pages later, and the newsletter
> will be automatically updated on the website through our standard website
> workflow. It also avoids the potential issue of the source google docs
> disappearing in the future, as they are stored in a community location.
>
> [1] https://github.com/apache/beam/tree/master/website/src/community
>
>
> On Wed, Mar 6, 2019 at 10:36 AM Rose Nguyen <rt...@google.com> wrote:
>
> I think that would be a great idea to change formats to help with
> distribution. I'm open to suggestions! I'm currently using a Google doc to
> collect and edit, then copy/paste sending the newsletter out directly, based
> on an interpretation of this discussion
> <https://lists.apache.org/thread.html/1f638eae43fe8abcb2f8752141c96d3dbdac86a583e0790044eea727@%3Cdev.beam.apache.org%3E>
> .
>
> How about this doc->website->Beam site workflow?:
>
> 1. The same usual newsletter [CALL FOR ITEMS] where you can contribute
> to the google doc, with soft deadlines for when I'll publish.
> 2. I'll publish the doc itself onto a website.
> 3. The newsletter is mailed out in the same way, but now with a
> shareable website link.
> 4. We'll keep an index of archived newsletter web pages on the Beam
> site, under the Community tab.
> 5. If you want to submit more content after the soft deadline, add it
> to the google doc and let me know to republish. I don't want to make the
> publication changes automatic because that leaves us open to tampering.
>
>
> This process is more laggy, so I'd suggest doing a 2 month vs monthly
> newsletter cadence. If we're happy with this idea, I'll send in a website
> PR for a new "Newsletter" left nav item under Community.
>
> Here's an example of a published newsletter: Apache Beam February-March
> 2019
> <https://gdoc.pub/doc/e/2PACX-1vTQIS4WkxV-HpgX5Lb6q05g4-wuIVcYd82123Mp4Y6q9fMv6Ynwd-l7dI4TrMyCrKilyU-YsoitbnZB>
>
>
> - This link is permanent unless the principal google doc is deleted.
> - Changes to the google doc after web publication are not
> automatically published on the website to protect the information integrity.
> - Republishing is quick and easy for me if you let me know you've
> added more.
> - I'll improve the formatting later if we go with this route.
>
> Any thoughts?
>
> On Wed, Mar 6, 2019 at 6:13 AM Thomas Weise <th...@apache.org> wrote:
>
> Similar to blog posts. A link that can be shared would also help to
> distribute over other channels, such as Twitter.
>
>
> On Wed, Mar 6, 2019, 6:06 AM Ismaël Mejía <ie...@gmail.com> wrote:
>
> We should have these newsletters published somewhere with a fixed URL so
> we can add missing updates, I have at least missed putting stuff in the
> last 3 ones just to realize when the newsletter has been already
> 'published' (and sadly interesting features like zstd have not been
> announced because of this).
>
> On Wed, Mar 6, 2019 at 2:48 PM Etienne Chauchot <ec...@apache.org>
> wrote:
>
> Hi,
> I would add in what's been done:
>
> Work on cassandraIO (Etienne Chauchot, Mathieu Blanchard, Frank Shahar) :
> refactorings, bugfixes, new where clause, security fix
>
> Etienne
>
> Le lundi 04 mars 2019 à 18:36 +0100, Suneel Marthi a écrit :
>
> Is this the final draft? - we had 2 beam talks at Big Data Tech Warsaw
> last Wednesday - I can send the updates offline.
>
> On Mon, Mar 4, 2019 at 6:16 PM Rose Nguyen <rt...@google.com> wrote:
>
>
> [image: Beam.png]
> February-March 2019 | Newsletter
>
> What’s been done
>
> ------------------------------
>
> Apache Beam 2.10.0 released (by: many contributors)
>
> - Download the release here.
> <https://beam.apache.org/get-started/downloads/>
> - See the blog post
> <https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html> for more
> details.
>
>
> Apache Beam awarded the 2019 Technology of the Year Award!
>
> - InfoWorld just awarded Beam the 2019 Technology of the Year Award.
> - See this article
> <https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html?nsdr=true>
> for more details.
>
>
> Kettle Beam 0.5 released with support for flink (by: Matt Casters)
>
> - Kettle now supports Apache Flink as well as Cloud Dataflow and Spark.
> - See Matt’s Blog
> <http://sandbox.kettle.be/wordpress/index.php/2019/02/24/kettle-beam-update-0-5-0/>
> for more details.
>
>
>
> What we’re working on...
>
> ------------------------------
>
> Apache Beam 2.11.0 release (by: many contributors)
>
>
> Hive Metastore Table provider for SQL (by: Anton Kedin)
>
> - Support for plugging table providers through Beam SQL API to allow
> obtaining table schemas from external sources.
> - See the PR <https://github.com/apache/beam/pull/7746> for more
> details.
>
>
> User Defined Coders for the Beam Go SDK (by: Robert Burke)
>
> - Working on expanding the variety of user defined types that can be a
> member of a PCollection in the Go SDK.
> - See BEAM-3306 <https://issues.apache.org/jira/browse/BEAM-3306> for
> more details.
>
>
> Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen, Mark Liu, Robbe
> Sneyders, Juta Staes, Valentyn Tymofieiev)
>
> - Beam 2.11.0 is the first release offering partial Python 3 support.
> - Many thanks to all contributors who helped to reach this milestone.
> - IO availablility on Python 3 is currently limited and only Python
> 3.5 version has been tested extensively.
> - Stay tuned on BEAM-1251 for more details.
>
>
> Notebooks for quickstarts and custom I/O (by: David Cavazos)
>
> - Adding IPython notebooks and snippets
> - See [BEAM-6557] <https://github.com/apache/beam/pull/7679> for more
> details.
>
>
>
>
> New members
> ------------------------------
>
> New PMC member!
>
> - Etienne Chauchot, Nantes, France
>
>
> New Committers!
>
> - Gleb Kanterov, Stockholm, Sweden
> - Michael Luckey
>
>
> New Contributors!
>
> - Kyle Weaver, San Francisco, CA
> - Would like to help begin implementing portability support for the
> Spark runner
> - Tanay Tummapalli, Delhi, India
> - Would like to contribute to Open Source this summer as part of
> Google Summer of Code
> - Brian Hulette, Seattle, WA
> - Contributing to Beam Portability
> - Michał Walenia, Warsaw, Poland
> - Working on integration and load testing
> - Daniel Chen, San Francisco, CA
> - Working on Beam Samza runner
>
>
>
> Talks & meetups
> ------------------------------
>
>
> Plugin Machine Intelligence and Apache Beam with Pentaho - Feb 7 @ London
>
> - Watch the How to Run Kettle on Apache Beam video here
> <https://skillsmatter.com/skillscasts/13405-how-to-run-kettle-on-apache-beam#video>.
>
> - See event details here
> <https://www.meetup.com/Pentaho-London-User-Group/events/256773962/>..
>
>
> Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @ San Francisco,
> CA
>
> - Organized by Thomas Weise and Austin Bennet, with speakers Tyler
> Akidau, Robert Crowe, Thomas Weise and Amar Pai
> - See event details here
> <https://www.meetup.com/San-Francisco-Apache-Beam/events/257482350/>
> and the slides for these presentation: Overview of Apache Beam and
> TensorFlow Transform (TFX) with Apache Beam
> <http://s.apache.org/beam-intro-feb-2019>, Python Streaming Pipelines
> with Beam on Flink <http://go.lyft.com/python-flink-beam-meetup-2019>, Dynamic
> pricing of Lyft rides using streaming
> <https://www.slideshare.net/AmarPai2/dynamic-pricing-of-lyft-rides-using-streaming>
>
> .
> Flink meetup - Feb 21@ Seattle, WA
>
> - Speakers from Alibaba, Google, and Uber gave talks about Apache
> Flink with Hive, Tensorflow, Beam, and AthenaX.
> - See event details here
> <https://www.meetup.com/seattle-flink/events/258723322/> and
> presentations here <https://www.slideshare.net/BowenLi9/presentations>.
>
>
>
> Beam Summit Europe 2019 - June 19-20 @ Berlin
>
> - Beam Summit Europe 2019 will take place in Berlin on June 19-20.
> - Speaker CfP and other details to follow soon!
> - Twitter announcement!
> <https://twitter.com/matthiasbaetens/status/1098854758893273088>
>
>
>
> Resources
> ------------------------------
>
> Apache Jira Beginner’s Guide (by: Daniel Oliveira)
>
> - A guide
> <https://cwiki.apache.org/confluence/display/BEAM/Beam+Jira+Beginner%27s+Guide>
> to introduce Beam contributors to the basics of using the Apache Jira for
> Beam development. Feedback welcomed!
>
>
> An approach to community building from Apache Beam (by: Kenn Knowles)
>
> - The Apache Software Foundation has published committer guidelines to
> help Beam's community building work.
> - See the post <https://blogs.apache.org/comdev/date/20190222> on the
> ASF blog.
>
>
> Exploring Beam SQL on Google Cloud Platform (by: Graham Polley)
>
> - “In this article, I’ll dive into this new feature of Beam, and see
> how it works by using a pipeline to read a data file from GCS, transform
> it, and then perform a basic calculation on the values contained in the
> file”.
> - See article
> <https://medium.com/weareservian/exploring-beam-sql-on-google-cloud-platform-b6c77f9b4af4>
> and full source code
> <https://github.com/polleyg/gcp-batch-ingestion-bigquery/blob/beam_sql/src/main/java/org/polleyg/BeamSQLMagic.java>
> .
>
>
> *Until Next Time!*
> --
> Rose Thị Nguyễn
>
>
>
> --
> Rose Thị Nguyễn
>
>
>
> --
> Rose Thị Nguyễn
>
>
>
> --
>
> *Aizhamal Nurmamat kyzy*
> Open Source Program Manager
> 646-355-9740 <(646)%20355-9740> Mobile
> 601 North 34th Street, Seattle, WA 98103
> <https://maps.google.com/?q=601+North+34th+Street,+Seattle,+WA+98103&entry=gmail&source=g>
>
>
>
>
>
Re: Apache Beam Newsletter - February/March 2019
Posted by Etienne Chauchot <ec...@apache.org>.
Hi,+1 on the process+1 on the new News section
I also think that matching the newsletter pace with the release pace makes it clearer for users. Also, as a general
obvious comment, newsletter have the interest of providing the ongoing work compared to release notes that provide only
the work that was done.
Etienne
Le mardi 12 mars 2019 à 21:50 -0700, Kenneth Knowles a écrit :
> On Tue, Mar 12, 2019 at 9:04 PM Thomas Weise <th...@apache.org> wrote:
> > The release blog is already on the radar for improvement [1].
> > I don't think there is a need to separate out the release blogs. That's if they provide content that is valuable to
> > users (IMO that's not exactly the case today).
>
> The case I am thinking about is a user searching the web for an issue and figuring out what version it was fixed in.
> Dep upgrades being a primary thing to notice.
> > If release blogs are just reformatted JIRA reports, then maybe we can skip them altogether (release notes are
> > already linked from download page).
>
> One reason is that Jira remains mutable and I am not so sure of the SEO, in terms of helping users figure out if an
> upgrade might solve their problem.
> > In that case I would much rather like to see the original JIRA report cleaned up as part of the release process.
>
> Either way, this seems worthwhile. I think release managers often do this anyhow. How could we formalize it? Do we
> need to? My favorite form of formal process is just to get more eyes on some LGTM.
> > On the other hand, if release blogs are assembled similar to the newsletter that we discuss here, through broader
> > contributor input and with the aim to provide more meaningful content to users, then I don't see why we need to put
> > them into a different area.
>
> Same answer: people searching for issue resolution. I wonder if our release notes should be a static copy/paste of the
> Jira list, deleting things that really make no sense and editing everything else to be meaningful. But I get the idea
> that we don't really need a separate area.
> What if newsletters matched releases, and could be drafted throughout the 6 week period, with folks really trying to
> give a narrative to what they contributed to a release?
>
> Kenn
> > Overall, given proposed newsletter cadence of 2 months and existing release cadence, we would probably still end up
> > with a monthly piece of news.
> >
> > Thanks,
> > Thomas
> >
> >
> >
https://lists.apache.org/thread.html/224908775d8807eecc53e45f14f99d6a439beb8bb5165e3a813bf82d@%3Cdev.beam.apache.org%3E
> >
> >
> > On Tue, Mar 12, 2019 at 9:27 AM Rose Nguyen <rt...@google.com> wrote:
> > > Once every 2 months works. We include both big and small items. It'll provide the focus Thomas mentions but still
> > > catches the frequency that Pablo suggested for relevancy. To Alexey's point, it's difficult to navigate the more
> > > recent non-release blog posts (<1 year) because they are sprinkled in.
> > >
> > > A compromise for all of our points is to move the release notes to a separate section, and have a single section
> > > that's blog/news.
> > >
> > > And thanks for wanting to contribute, Aizhamal! Teaming up with you will make things run smoothly.
> > >
> > >
> > > On Tue, Mar 12, 2019 at 3:58 AM Alexey Romanenko <ar...@gmail.com> wrote:
> > > > +1 for “News” section. Though, I’d propose to exclude “release notes” posts from “Blog” section list (since now
> > > > we have quite regular releases, it makes blog posts list not very readable for users) and move them to new
> > > > created News section. Also, this section could include the posts about other Beam events, like meet-ups,
> > > > conferences, project updates, etc. I’d keep “Blog” section for more technical posts.
> > > >
> > > > > On 12 Mar 2019, at 06:31, Pablo Estrada <pa...@google.com> wrote:
> > > > >
> > > > > I agree that the newsletter fits well as a blog post. I think that'd work best.
> > > > > As for the cadence, I think quarterly would be a bit too infrequent. I like once a month, or once every other
> > > > > month to have at least one per release. Though happy to hear other people's thoughts.
> > > > > Best
> > > > > -P.
> > > > > On Mon, Mar 11, 2019 at 6:42 PM Thomas Weise <th...@apache.org> wrote:
> > > > > > +1 for single blog/news section
> > > > > >
> > > > > > Also I wouldn't mind quarterly cadence, to provide more focus for folks to contribute.
> > > > > > On Mon, Mar 11, 2019 at 6:35 PM Kenneth Knowles <ke...@apache.org> wrote:
> > > > > > > Could the newsletter be a blog entry? If you check out https://blogs.apache.org/ many of the posts are
> > > > > > > "Apache News Round-up". We could rename the "Blog" section to "News" if you ask me.
> > > > > > >
> > > > > > > Kenn
> > > > > > > On Mon, Mar 11, 2019 at 5:13 PM Aizhamal Nurmamat kyzy <ai...@google.com> wrote:
> > > > > > > > Hello everyone,
> > > > > > > > I had a chat with Rose on how I can support the effort and keep sending the newsletters on a monthly
> > > > > > > > basis. The new workflow would look as follows:
> > > > > > > > Send out the same [CALL FOR ITEMS] where you can contribute to the Google doc with deadlinesEdit the doc
> > > > > > > > after the deadlineConvert the file into MarkdownSend in PR to add the file to Beam repo in Newsletter
> > > > > > > > directoryHave people send their fixes/updates through PRsIn this effort, I can support Rose in steps 3 &
> > > > > > > > 4.
> > > > > > > >
> > > > > > > > We would also need:
> > > > > > > > Create a Newsletter section under the Community tabWrite guidelines on newsletter contributionsMake a
> > > > > > > > note about timing e.g. if upcoming event, then add to the next newsletterHow does that sound to you all?
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Mar 6, 2019 at 9:19 PM Rose Nguyen <rt...@google.com> wrote:
> > > > > > > > > Good points. With the suggested workflow I think I can support a quarterly newsletter. I'm also happy
> > > > > > > > > to get more involvement from others to do this work and we can see what cadence that allows.
> > > > > > > > > On Wed, Mar 6, 2019 at 8:22 PM Kenneth Knowles <ke...@apache.org> wrote:
> > > > > > > > > > Good points Melissa & Austin.
> > > > > > > > > > - archive in the repo & on the website
> > > > > > > > > > - put missed items on the next newsletter, so anyone following sees them
> > > > > > > > > >
> > > > > > > > > > Kenn
> > > > > > > > > > On Wed, Mar 6, 2019 at 3:26 PM Suneel Marthi <sm...@apache.org> wrote:
> > > > > > > > > > > I believe there was also a Beam workshop or working session in Warsaw last week.
> > > > > > > > > > > On Wed, Mar 6, 2019 at 6:20 PM Austin Bennett <wh...@gmail.com> wrote:
> > > > > > > > > > > > +1 for archive in our repo.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > I do follow the newsletter, but am unlikely to go back and look into the past for
> > > > > > > > > > > > changes/updates.
> > > > > > > > > > > > Would suggest that things that get missed in one newsletter (a concrete example, Suneel's talks
> > > > > > > > > > > > not mentioned in the newsletter) would get published in the next iteration, rather than editing
> > > > > > > > > > > > the past 'published' newsletter. Put another way, save editing the past for corrections (typos,
> > > > > > > > > > > > things being incorrect). Else, I imagine that I'm unlikely to catch a great announcement that
> > > > > > > > > > > > warranted being in the newsletter in the first place. This certainly works better with a
> > > > > > > > > > > > regular/frequent release cadence, like we arrived at for version releases (then, if something
> > > > > > > > > > > > misses one cut, it is not too big a deal, as the next release is coming soon).
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > On Wed, Mar 6, 2019 at 12:50 PM Melissa Pashniak <me...@google.com> wrote:
> > > > > > > > > > > > > For step #2 (publishing onto the website), I think it would be good to stay consistent with
> > > > > > > > > > > > > our existing workflows if possible. Rather than using an external tool, what about:
> > > > > > > > > > > > >
> > > > > > > > > > > > > After a google doc newsletter draft is ready, convert it into a standard markdown file and put
> > > > > > > > > > > > > it into our GitHub repo, perhaps in a new newsletter directory in the website community
> > > > > > > > > > > > > directory [1]. These would be listed for browsing on a Newsletters page as mentioned in step
> > > > > > > > > > > > > #4. People can then just open a PR to add missing things to the pages later, and the
> > > > > > > > > > > > > newsletter will be automatically updated on the website through our standard website workflow.
> > > > > > > > > > > > > It also avoids the potential issue of the source google docs disappearing in the future, as
> > > > > > > > > > > > > they are stored in a community location.
> > > > > > > > > > > > >
> > > > > > > > > > > > > [1] https://github.com/apache/beam/tree/master/website/src/community
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Wed, Mar 6, 2019 at 10:36 AM Rose Nguyen <rt...@google.com> wrote:
> > > > > > > > > > > > > > I think that would be a great idea to change formats to help with distribution. I'm open to
> > > > > > > > > > > > > > suggestions! I'm currently using a Google doc to collect and edit, then copy/paste sending
> > > > > > > > > > > > > > the newsletter out directly, based on an interpretation of this discussion.
> > > > > > > > > > > > > > How about this doc->website->Beam site workflow?:
> > > > > > > > > > > > > > The same usual newsletter [CALL FOR ITEMS] where you can contribute to the google doc, with
> > > > > > > > > > > > > > soft deadlines for when I'll publish.I'll publish the doc itself onto a website.The
> > > > > > > > > > > > > > newsletter is mailed out in the same way, but now with a shareable website link.We'll keep
> > > > > > > > > > > > > > an index of archived newsletter web pages on the Beam site, under the Community tab.If you
> > > > > > > > > > > > > > want to submit more content after the soft deadline, add it to the google doc and let me
> > > > > > > > > > > > > > know to republish. I don't want to make the publication changes automatic because that
> > > > > > > > > > > > > > leaves us open to tampering.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > This process is more laggy, so I'd suggest doing a 2 month vs monthly newsletter cadence. If
> > > > > > > > > > > > > > we're happy with this idea, I'll send in a website PR for a new "Newsletter" left nav item
> > > > > > > > > > > > > > under Community.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Here's an example of a published newsletter: Apache Beam February-March 2019
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > This link is permanent unless the principal google doc is deleted. Changes to the google doc
> > > > > > > > > > > > > > after web publication are not automatically published on the website to protect the
> > > > > > > > > > > > > > information integrity.Republishing is quick and easy for me if you let me know you've added
> > > > > > > > > > > > > > more.I'll improve the formatting later if we go with this route.Any thoughts?
> > > > > > > > > > > > > > On Wed, Mar 6, 2019 at 6:13 AM Thomas Weise <th...@apache.org> wrote:
> > > > > > > > > > > > > > > Similar to blog posts. A link that can be shared would also help to distribute over other
> > > > > > > > > > > > > > > channels, such as Twitter.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Wed, Mar 6, 2019, 6:06 AM Ismaël Mejía <ie...@gmail.com> wrote:
> > > > > > > > > > > > > > > > We should have these newsletters published somewhere with a fixed URL so we can add
> > > > > > > > > > > > > > > > missing updates, I have at least missed putting stuff in the last 3 ones just to realize
> > > > > > > > > > > > > > > > when the newsletter has been already 'published' (and sadly interesting features like
> > > > > > > > > > > > > > > > zstd have not been announced because of this).
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > On Wed, Mar 6, 2019 at 2:48 PM Etienne Chauchot <ec...@apache.org> wrote:
> > > > > > > > > > > > > > > > > Hi,I would add in what's been done:
> > > > > > > > > > > > > > > > > Work on cassandraIO (Etienne Chauchot, Mathieu Blanchard, Frank Shahar) :
> > > > > > > > > > > > > > > > > refactorings, bugfixes, new where clause, security fix
> > > > > > > > > > > > > > > > > Etienne
> > > > > > > > > > > > > > > > > Le lundi 04 mars 2019 à 18:36 +0100, Suneel Marthi a écrit :
> > > > > > > > > > > > > > > > > > Is this the final draft? - we had 2 beam talks at Big Data Tech Warsaw last
> > > > > > > > > > > > > > > > > > Wednesday - I can send the updates offline.
> > > > > > > > > > > > > > > > > > On Mon, Mar 4, 2019 at 6:16 PM Rose Nguyen <rt...@google.com> wrote:
> > > > > > > > > > > > > > > > > > > February-March 2019 | Newsletter
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > What’s been done
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Apache Beam 2.10.0 released (by: many contributors)
> > > > > > > > > > > > > > > > > > > Download the release here.See the blog post for more details.
> > > > > > > > > > > > > > > > > > > Apache Beam awarded the 2019 Technology of the Year Award!
> > > > > > > > > > > > > > > > > > > InfoWorld just awarded Beam the 2019 Technology of the Year Award.See this
> > > > > > > > > > > > > > > > > > > article for more details.
> > > > > > > > > > > > > > > > > > > Kettle Beam 0.5 released with support for flink (by: Matt Casters)
> > > > > > > > > > > > > > > > > > > Kettle now supports Apache Flink as well as Cloud Dataflow and Spark.See Matt’s
> > > > > > > > > > > > > > > > > > > Blog for more details.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > What we’re working on...
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Apache Beam 2.11.0 release (by: many contributors)
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Hive Metastore Table provider for SQL (by: Anton Kedin)
> > > > > > > > > > > > > > > > > > > Support for plugging table providers through Beam SQL API to allow obtaining table
> > > > > > > > > > > > > > > > > > > schemas from external sources.See the PR for more details.
> > > > > > > > > > > > > > > > > > > User Defined Coders for the Beam Go SDK (by: Robert Burke)
> > > > > > > > > > > > > > > > > > > Working on expanding the variety of user defined types that can be a member of a
> > > > > > > > > > > > > > > > > > > PCollection in the Go SDK.See BEAM-3306 for more details.
> > > > > > > > > > > > > > > > > > > Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen, Mark Liu, Robbe
> > > > > > > > > > > > > > > > > > > Sneyders, Juta Staes, Valentyn Tymofieiev)
> > > > > > > > > > > > > > > > > > > Beam 2.11.0 is the first release offering partial Python 3 support. Many thanks to
> > > > > > > > > > > > > > > > > > > all contributors who helped to reach this milestone.IO availablility on Python 3
> > > > > > > > > > > > > > > > > > > is currently limited and only Python 3.5 version has been tested extensively. Stay
> > > > > > > > > > > > > > > > > > > tuned on BEAM-1251 for more details.
> > > > > > > > > > > > > > > > > > > Notebooks for quickstarts and custom I/O (by: David Cavazos)
> > > > > > > > > > > > > > > > > > > Adding IPython notebooks and snippetsSee [BEAM-6557] for more details.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > New members
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > New PMC member!
> > > > > > > > > > > > > > > > > > > Etienne Chauchot, Nantes, France
> > > > > > > > > > > > > > > > > > > New Committers!
> > > > > > > > > > > > > > > > > > > Gleb Kanterov, Stockholm, SwedenMichael Luckey
> > > > > > > > > > > > > > > > > > > New Contributors!
> > > > > > > > > > > > > > > > > > > Kyle Weaver, San Francisco, CAWould like to help begin implementing portability
> > > > > > > > > > > > > > > > > > > support for the Spark runnerTanay Tummapalli, Delhi, IndiaWould like to contribute
> > > > > > > > > > > > > > > > > > > to Open Source this summer as part of Google Summer of CodeBrian Hulette, Seattle,
> > > > > > > > > > > > > > > > > > > WAContributing to Beam PortabilityMichał Walenia, Warsaw, PolandWorking on
> > > > > > > > > > > > > > > > > > > integration and load testing Daniel Chen, San Francisco, CAWorking on Beam Samza
> > > > > > > > > > > > > > > > > > > runner
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Talks & meetups
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Plugin Machine Intelligence and Apache Beam with Pentaho - Feb 7 @ London
> > > > > > > > > > > > > > > > > > > Watch the How to Run Kettle on Apache Beam video here. See event details here..
> > > > > > > > > > > > > > > > > > > Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @ San Francisco, CA
> > > > > > > > > > > > > > > > > > > Organized by Thomas Weise and Austin Bennet, with speakers Tyler Akidau, Robert
> > > > > > > > > > > > > > > > > > > Crowe, Thomas Weise and Amar PaiSee event details here and the slides for these
> > > > > > > > > > > > > > > > > > > presentation: Overview of Apache Beam and TensorFlow Transform (TFX) with Apache
> > > > > > > > > > > > > > > > > > > Beam, Python Streaming Pipelines with Beam on Flink, Dynamic pricing of Lyft rides
> > > > > > > > > > > > > > > > > > > using streaming.
> > > > > > > > > > > > > > > > > > > Flink meetup - Feb 21@ Seattle, WA
> > > > > > > > > > > > > > > > > > > Speakers from Alibaba, Google, and Uber gave talks about Apache Flink with Hive,
> > > > > > > > > > > > > > > > > > > Tensorflow, Beam, and AthenaX.See event details here and presentations here.
> > > > > > > > > > > > > > > > > > > Beam Summit Europe 2019 - June 19-20 @ Berlin
> > > > > > > > > > > > > > > > > > > Beam Summit Europe 2019 will take place in Berlin on June 19-20.Speaker CfP and
> > > > > > > > > > > > > > > > > > > other details to follow soon!Twitter announcement!
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Resources
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Apache Jira Beginner’s Guide (by: Daniel Oliveira)
> > > > > > > > > > > > > > > > > > > A guide to introduce Beam contributors to the basics of using the Apache Jira for
> > > > > > > > > > > > > > > > > > > Beam development. Feedback welcomed!
> > > > > > > > > > > > > > > > > > > An approach to community building from Apache Beam (by: Kenn Knowles)
> > > > > > > > > > > > > > > > > > > The Apache Software Foundation has published committer guidelines to help Beam's
> > > > > > > > > > > > > > > > > > > community building work.See the post on the ASF blog.
> > > > > > > > > > > > > > > > > > > Exploring Beam SQL on Google Cloud Platform (by: Graham Polley)
> > > > > > > > > > > > > > > > > > > “In this article, I’ll dive into this new feature of Beam, and see how it works by
> > > > > > > > > > > > > > > > > > > using a pipeline to read a data file from GCS, transform it, and then perform a
> > > > > > > > > > > > > > > > > > > basic calculation on the values contained in the file”.See article and full source
> > > > > > > > > > > > > > > > > > > code.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Until Next Time!--
> > > > > > > > > > > > > > > > > > > Rose Thị Nguyễn
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > --
> > > > > > > > > > > > > > Rose Thị Nguyễn
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Rose Thị Nguyễn
> > > > > > > >
> > > > > > > > --
> > > > > > > >
> > > > > > > > Aizhamal Nurmamat kyzy
> > > > > > > > Open Source Program Manager
> > > > > > > > 646-355-9740 Mobile
> > > > > > > > 601 North 34th Street, Seattle, WA 98103
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > >
> > >
Re: Apache Beam Newsletter - February/March 2019
Posted by Kenneth Knowles <ke...@apache.org>.
On Tue, Mar 12, 2019 at 9:04 PM Thomas Weise <th...@apache.org> wrote:
> The release blog is already on the radar for improvement [1].
>
> I don't think there is a need to separate out the release blogs. That's if
> they provide content that is valuable to users (IMO that's not exactly the
> case today).
>
The case I am thinking about is a user searching the web for an issue and
figuring out what version it was fixed in. Dep upgrades being a primary
thing to notice.
> If release blogs are just reformatted JIRA reports, then maybe we can skip
> them altogether (release notes are already linked from download page).
>
One reason is that Jira remains mutable and I am not so sure of the SEO, in
terms of helping users figure out if an upgrade might solve their problem.
> In that case I would much rather like to see the original JIRA report
> cleaned up as part of the release process.
>
Either way, this seems worthwhile. I think release managers often do this
anyhow. How could we formalize it? Do we need to? My favorite form of
formal process is just to get more eyes on some LGTM.
> On the other hand, if release blogs are assembled similar to the
> newsletter that we discuss here, through broader contributor input and with
> the aim to provide more meaningful content to users, then I don't see why
> we need to put them into a different area.
>
Same answer: people searching for issue resolution. I wonder if our release
notes should be a static copy/paste of the Jira list, deleting things that
really make no sense and editing everything else to be meaningful. But I
get the idea that we don't really need a separate area.
What if newsletters matched releases, and could be drafted throughout the 6
week period, with folks really trying to give a narrative to what they
contributed to a release?
Kenn
Overall, given proposed newsletter cadence of 2 months and existing release
> cadence, we would probably still end up with a monthly piece of news.
>
> Thanks,
> Thomas
>
>
>
> https://lists.apache.org/thread.html/224908775d8807eecc53e45f14f99d6a439beb8bb5165e3a813bf82d@%3Cdev.beam.apache.org%3E
>
>
> On Tue, Mar 12, 2019 at 9:27 AM Rose Nguyen <rt...@google.com> wrote:
>
>> Once every 2 months works. We include both big and small items. It'll
>> provide the focus Thomas mentions but still catches the frequency that
>> Pablo suggested for relevancy. To Alexey's point, it's difficult to
>> navigate the more recent non-release blog posts (<1 year) because they are
>> sprinkled in.
>>
>> A compromise for all of our points is to move the release notes to a
>> separate section, and have a single section that's blog/news.
>>
>> And thanks for wanting to contribute, Aizhamal! Teaming up with you will
>> make things run smoothly.
>>
>> On Tue, Mar 12, 2019 at 3:58 AM Alexey Romanenko <
>> aromanenko.dev@gmail.com> wrote:
>>
>>> +1 for “News” section. Though, I’d propose to exclude “release notes”
>>> posts from “Blog” section list (since now we have quite regular releases,
>>> it makes blog posts list not very readable for users) and move them to new
>>> created News section. Also, this section could include the posts about
>>> other Beam events, like meet-ups, conferences, project updates, etc. I’d
>>> keep “Blog” section for more technical posts.
>>>
>>> On 12 Mar 2019, at 06:31, Pablo Estrada <pa...@google.com> wrote:
>>>
>>> I agree that the newsletter fits well as a blog post. I think that'd
>>> work best.
>>>
>>> As for the cadence, I think quarterly would be a bit too infrequent. I
>>> like once a month, or once every other month to have at least one per
>>> release. Though happy to hear other people's thoughts.
>>> Best
>>> -P.
>>>
>>> On Mon, Mar 11, 2019 at 6:42 PM Thomas Weise <th...@apache.org> wrote:
>>>
>>>> +1 for single blog/news section
>>>>
>>>> Also I wouldn't mind quarterly cadence, to provide more focus for folks
>>>> to contribute.
>>>>
>>>> On Mon, Mar 11, 2019 at 6:35 PM Kenneth Knowles <ke...@apache.org>
>>>> wrote:
>>>>
>>>>> Could the newsletter be a blog entry? If you check out
>>>>> https://blogs.apache.org/ many of the posts are "Apache News
>>>>> Round-up". We could rename the "Blog" section to "News" if you ask me.
>>>>>
>>>>> Kenn
>>>>>
>>>>> On Mon, Mar 11, 2019 at 5:13 PM Aizhamal Nurmamat kyzy <
>>>>> aizhamal@google.com> wrote:
>>>>>
>>>>>> Hello everyone,
>>>>>>
>>>>>> I had a chat with Rose on how I can support the effort and keep
>>>>>> sending the newsletters on a monthly basis. The new workflow would look as
>>>>>> follows:
>>>>>>
>>>>>> 1. Send out the same [CALL FOR ITEMS] where you can contribute to
>>>>>> the Google doc with deadlines
>>>>>> 2. Edit the doc after the deadline
>>>>>> 3. Convert the file into Markdown
>>>>>> 4. Send in PR to add the file to Beam repo in Newsletter directory
>>>>>> 5. Have people send their fixes/updates through PRs
>>>>>>
>>>>>> In this effort, I can support Rose in steps 3 & 4.
>>>>>>
>>>>>> We would also need:
>>>>>>
>>>>>> - Create a Newsletter section under the Community tab
>>>>>> - Write guidelines on newsletter contributions
>>>>>> - Make a note about timing e.g. if upcoming event, then add to
>>>>>> the next newsletter
>>>>>>
>>>>>> How does that sound to you all?
>>>>>>
>>>>>>
>>>>>> On Wed, Mar 6, 2019 at 9:19 PM Rose Nguyen <rt...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Good points. With the suggested workflow I think I can support a
>>>>>>> quarterly newsletter. I'm also happy to get more involvement from others to
>>>>>>> do this work and we can see what cadence that allows.
>>>>>>>
>>>>>>> On Wed, Mar 6, 2019 at 8:22 PM Kenneth Knowles <ke...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Good points Melissa & Austin.
>>>>>>>>
>>>>>>>> - archive in the repo & on the website
>>>>>>>> - put missed items on the next newsletter, so anyone following
>>>>>>>> sees them
>>>>>>>>
>>>>>>>> Kenn
>>>>>>>>
>>>>>>>> On Wed, Mar 6, 2019 at 3:26 PM Suneel Marthi <sm...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I believe there was also a Beam workshop or working session in
>>>>>>>>> Warsaw last week.
>>>>>>>>>
>>>>>>>>> On Wed, Mar 6, 2019 at 6:20 PM Austin Bennett <
>>>>>>>>> whatwouldaustindo@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> +1 for archive in our repo.
>>>>>>>>>>
>>>>>>>>>> I do follow the newsletter, but am unlikely to go back and look
>>>>>>>>>> into the past for changes/updates.
>>>>>>>>>>
>>>>>>>>>> Would suggest that things that get missed in one newsletter (a
>>>>>>>>>> concrete example, Suneel's talks not mentioned in the newsletter) would get
>>>>>>>>>> published in the next iteration, rather than editing the past 'published'
>>>>>>>>>> newsletter. Put another way, save editing the past for corrections (typos,
>>>>>>>>>> things being incorrect). Else, I imagine that I'm unlikely to catch a
>>>>>>>>>> great announcement that warranted being in the newsletter in the first
>>>>>>>>>> place. This certainly works better with a regular/frequent release
>>>>>>>>>> cadence, like we arrived at for version releases (then, if something misses
>>>>>>>>>> one cut, it is not too big a deal, as the next release is coming soon).
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Mar 6, 2019 at 12:50 PM Melissa Pashniak <
>>>>>>>>>> melissapa@google.com> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> For step #2 (publishing onto the website), I think it would be
>>>>>>>>>>> good to stay consistent with our existing workflows if possible. Rather
>>>>>>>>>>> than using an external tool, what about:
>>>>>>>>>>>
>>>>>>>>>>> After a google doc newsletter draft is ready, convert it into a
>>>>>>>>>>> standard markdown file and put it into our GitHub repo, perhaps in a new
>>>>>>>>>>> newsletter directory in the website community directory [1]. These would be
>>>>>>>>>>> listed for browsing on a Newsletters page as mentioned in step #4. People
>>>>>>>>>>> can then just open a PR to add missing things to the pages later, and the
>>>>>>>>>>> newsletter will be automatically updated on the website through our
>>>>>>>>>>> standard website workflow. It also avoids the potential issue of the source
>>>>>>>>>>> google docs disappearing in the future, as they are stored in a community
>>>>>>>>>>> location.
>>>>>>>>>>>
>>>>>>>>>>> [1]
>>>>>>>>>>> https://github.com/apache/beam/tree/master/website/src/community
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Mar 6, 2019 at 10:36 AM Rose Nguyen <rt...@google.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I think that would be a great idea to change formats to help
>>>>>>>>>>>> with distribution. I'm open to suggestions! I'm currently using a Google
>>>>>>>>>>>> doc to collect and edit, then copy/paste sending the newsletter out
>>>>>>>>>>>> directly, based on an interpretation of this discussion
>>>>>>>>>>>> <https://lists.apache.org/thread.html/1f638eae43fe8abcb2f8752141c96d3dbdac86a583e0790044eea727@%3Cdev.beam.apache.org%3E>
>>>>>>>>>>>> .
>>>>>>>>>>>>
>>>>>>>>>>>> How about this doc->website->Beam site workflow?:
>>>>>>>>>>>>
>>>>>>>>>>>> 1. The same usual newsletter [CALL FOR ITEMS] where you can
>>>>>>>>>>>> contribute to the google doc, with soft deadlines for when I'll publish.
>>>>>>>>>>>> 2. I'll publish the doc itself onto a website.
>>>>>>>>>>>> 3. The newsletter is mailed out in the same way, but now
>>>>>>>>>>>> with a shareable website link.
>>>>>>>>>>>> 4. We'll keep an index of archived newsletter web pages on
>>>>>>>>>>>> the Beam site, under the Community tab.
>>>>>>>>>>>> 5. If you want to submit more content after the soft
>>>>>>>>>>>> deadline, add it to the google doc and let me know to republish. I don't
>>>>>>>>>>>> want to make the publication changes automatic because that leaves us open
>>>>>>>>>>>> to tampering.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> This process is more laggy, so I'd suggest doing a 2 month vs
>>>>>>>>>>>> monthly newsletter cadence. If we're happy with this idea, I'll send in a
>>>>>>>>>>>> website PR for a new "Newsletter" left nav item under Community.
>>>>>>>>>>>>
>>>>>>>>>>>> Here's an example of a published newsletter: Apache Beam
>>>>>>>>>>>> February-March 2019
>>>>>>>>>>>> <https://gdoc.pub/doc/e/2PACX-1vTQIS4WkxV-HpgX5Lb6q05g4-wuIVcYd82123Mp4Y6q9fMv6Ynwd-l7dI4TrMyCrKilyU-YsoitbnZB>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> - This link is permanent unless the principal google doc is
>>>>>>>>>>>> deleted.
>>>>>>>>>>>> - Changes to the google doc after web publication are not
>>>>>>>>>>>> automatically published on the website to protect the information integrity.
>>>>>>>>>>>> - Republishing is quick and easy for me if you let me know
>>>>>>>>>>>> you've added more.
>>>>>>>>>>>> - I'll improve the formatting later if we go with this
>>>>>>>>>>>> route.
>>>>>>>>>>>>
>>>>>>>>>>>> Any thoughts?
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Mar 6, 2019 at 6:13 AM Thomas Weise <th...@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Similar to blog posts. A link that can be shared would also
>>>>>>>>>>>>> help to distribute over other channels, such as Twitter.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Mar 6, 2019, 6:06 AM Ismaël Mejía <ie...@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> We should have these newsletters published somewhere with a
>>>>>>>>>>>>>> fixed URL so we can add missing updates, I have at least missed putting
>>>>>>>>>>>>>> stuff in the last 3 ones just to realize when the newsletter has been
>>>>>>>>>>>>>> already 'published' (and sadly interesting features like zstd have not been
>>>>>>>>>>>>>> announced because of this).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Mar 6, 2019 at 2:48 PM Etienne Chauchot <
>>>>>>>>>>>>>> echauchot@apache.org> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>> I would add in what's been done:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Work on cassandraIO (Etienne Chauchot, Mathieu Blanchard,
>>>>>>>>>>>>>>> Frank Shahar) : refactorings, bugfixes, new where clause, security fix
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Etienne
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Le lundi 04 mars 2019 à 18:36 +0100, Suneel Marthi a écrit :
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Is this the final draft? - we had 2 beam talks at Big Data
>>>>>>>>>>>>>>> Tech Warsaw last Wednesday - I can send the updates offline.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Mar 4, 2019 at 6:16 PM Rose Nguyen <
>>>>>>>>>>>>>>> rtnguyen@google.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [image: Beam.png]
>>>>>>>>>>>>>>> February-March 2019 | Newsletter
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> What’s been done
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Apache Beam 2.10.0 released (by: many contributors)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - Download the release here.
>>>>>>>>>>>>>>> <https://beam.apache.org/get-started/downloads/>
>>>>>>>>>>>>>>> - See the blog post
>>>>>>>>>>>>>>> <https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html>
>>>>>>>>>>>>>>> for more details.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Apache Beam awarded the 2019 Technology of the Year Award!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - InfoWorld just awarded Beam the 2019 Technology of the
>>>>>>>>>>>>>>> Year Award.
>>>>>>>>>>>>>>> - See this article
>>>>>>>>>>>>>>> <https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html?nsdr=true>
>>>>>>>>>>>>>>> for more details.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Kettle Beam 0.5 released with support for flink (by: Matt
>>>>>>>>>>>>>>> Casters)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - Kettle now supports Apache Flink as well as Cloud
>>>>>>>>>>>>>>> Dataflow and Spark.
>>>>>>>>>>>>>>> - See Matt’s Blog
>>>>>>>>>>>>>>> <http://sandbox.kettle.be/wordpress/index.php/2019/02/24/kettle-beam-update-0-5-0/>
>>>>>>>>>>>>>>> for more details.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> What we’re working on...
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Apache Beam 2.11.0 release (by: many contributors)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hive Metastore Table provider for SQL (by: Anton Kedin)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - Support for plugging table providers through Beam SQL
>>>>>>>>>>>>>>> API to allow obtaining table schemas from external sources.
>>>>>>>>>>>>>>> - See the PR <https://github.com/apache/beam/pull/7746>
>>>>>>>>>>>>>>> for more details.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> User Defined Coders for the Beam Go SDK (by: Robert Burke)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - Working on expanding the variety of user defined types
>>>>>>>>>>>>>>> that can be a member of a PCollection in the Go SDK.
>>>>>>>>>>>>>>> - See BEAM-3306
>>>>>>>>>>>>>>> <https://issues.apache.org/jira/browse/BEAM-3306> for
>>>>>>>>>>>>>>> more details.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen,
>>>>>>>>>>>>>>> Mark Liu, Robbe Sneyders, Juta Staes, Valentyn Tymofieiev)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - Beam 2.11.0 is the first release offering partial
>>>>>>>>>>>>>>> Python 3 support.
>>>>>>>>>>>>>>> - Many thanks to all contributors who helped to reach
>>>>>>>>>>>>>>> this milestone.
>>>>>>>>>>>>>>> - IO availablility on Python 3 is currently limited and
>>>>>>>>>>>>>>> only Python 3.5 version has been tested extensively.
>>>>>>>>>>>>>>> - Stay tuned on BEAM-1251 for more details.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Notebooks for quickstarts and custom I/O (by: David Cavazos)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - Adding IPython notebooks and snippets
>>>>>>>>>>>>>>> - See [BEAM-6557]
>>>>>>>>>>>>>>> <https://github.com/apache/beam/pull/7679> for more
>>>>>>>>>>>>>>> details.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> New members
>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> New PMC member!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - Etienne Chauchot, Nantes, France
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> New Committers!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - Gleb Kanterov, Stockholm, Sweden
>>>>>>>>>>>>>>> - Michael Luckey
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> New Contributors!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - Kyle Weaver, San Francisco, CA
>>>>>>>>>>>>>>> - Would like to help begin implementing portability
>>>>>>>>>>>>>>> support for the Spark runner
>>>>>>>>>>>>>>> - Tanay Tummapalli, Delhi, India
>>>>>>>>>>>>>>> - Would like to contribute to Open Source this summer as
>>>>>>>>>>>>>>> part of Google Summer of Code
>>>>>>>>>>>>>>> - Brian Hulette, Seattle, WA
>>>>>>>>>>>>>>> - Contributing to Beam Portability
>>>>>>>>>>>>>>> - Michał Walenia, Warsaw, Poland
>>>>>>>>>>>>>>> - Working on integration and load testing
>>>>>>>>>>>>>>> - Daniel Chen, San Francisco, CA
>>>>>>>>>>>>>>> - Working on Beam Samza runner
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Talks & meetups
>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Plugin Machine Intelligence and Apache Beam with Pentaho -
>>>>>>>>>>>>>>> Feb 7 @ London
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - Watch the How to Run Kettle on Apache Beam video here
>>>>>>>>>>>>>>> <https://skillsmatter.com/skillscasts/13405-how-to-run-kettle-on-apache-beam#video>.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - See event details here
>>>>>>>>>>>>>>> <https://www.meetup.com/Pentaho-London-User-Group/events/256773962/>
>>>>>>>>>>>>>>> ..
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @
>>>>>>>>>>>>>>> San Francisco, CA
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - Organized by Thomas Weise and Austin Bennet, with
>>>>>>>>>>>>>>> speakers Tyler Akidau, Robert Crowe, Thomas Weise and Amar Pai
>>>>>>>>>>>>>>> - See event details here
>>>>>>>>>>>>>>> <https://www.meetup.com/San-Francisco-Apache-Beam/events/257482350/>
>>>>>>>>>>>>>>> and the slides for these presentation: Overview of
>>>>>>>>>>>>>>> Apache Beam and TensorFlow Transform (TFX) with Apache Beam
>>>>>>>>>>>>>>> <http://s.apache.org/beam-intro-feb-2019>, Python
>>>>>>>>>>>>>>> Streaming Pipelines with Beam on Flink
>>>>>>>>>>>>>>> <http://go.lyft.com/python-flink-beam-meetup-2019>, Dynamic
>>>>>>>>>>>>>>> pricing of Lyft rides using streaming
>>>>>>>>>>>>>>> <https://www.slideshare.net/AmarPai2/dynamic-pricing-of-lyft-rides-using-streaming>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>> Flink meetup - Feb 21@ Seattle, WA
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - Speakers from Alibaba, Google, and Uber gave talks
>>>>>>>>>>>>>>> about Apache Flink with Hive, Tensorflow, Beam, and AthenaX.
>>>>>>>>>>>>>>> - See event details here
>>>>>>>>>>>>>>> <https://www.meetup.com/seattle-flink/events/258723322/>
>>>>>>>>>>>>>>> and presentations here
>>>>>>>>>>>>>>> <https://www.slideshare.net/BowenLi9/presentations>.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Beam Summit Europe 2019 - June 19-20 @ Berlin
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - Beam Summit Europe 2019 will take place in Berlin on
>>>>>>>>>>>>>>> June 19-20.
>>>>>>>>>>>>>>> - Speaker CfP and other details to follow soon!
>>>>>>>>>>>>>>> - Twitter announcement!
>>>>>>>>>>>>>>> <https://twitter.com/matthiasbaetens/status/1098854758893273088>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Resources
>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Apache Jira Beginner’s Guide (by: Daniel Oliveira)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - A guide
>>>>>>>>>>>>>>> <https://cwiki.apache.org/confluence/display/BEAM/Beam+Jira+Beginner%27s+Guide>
>>>>>>>>>>>>>>> to introduce Beam contributors to the basics of using the Apache Jira for
>>>>>>>>>>>>>>> Beam development. Feedback welcomed!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> An approach to community building from Apache Beam (by:
>>>>>>>>>>>>>>> Kenn Knowles)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - The Apache Software Foundation has published committer
>>>>>>>>>>>>>>> guidelines to help Beam's community building work.
>>>>>>>>>>>>>>> - See the post
>>>>>>>>>>>>>>> <https://blogs.apache.org/comdev/date/20190222> on the
>>>>>>>>>>>>>>> ASF blog.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Exploring Beam SQL on Google Cloud Platform (by: Graham
>>>>>>>>>>>>>>> Polley)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - “In this article, I’ll dive into this new feature of
>>>>>>>>>>>>>>> Beam, and see how it works by using a pipeline to read a data file from
>>>>>>>>>>>>>>> GCS, transform it, and then perform a basic calculation on the values
>>>>>>>>>>>>>>> contained in the file”.
>>>>>>>>>>>>>>> - See article
>>>>>>>>>>>>>>> <https://medium.com/weareservian/exploring-beam-sql-on-google-cloud-platform-b6c77f9b4af4>
>>>>>>>>>>>>>>> and full source code
>>>>>>>>>>>>>>> <https://github.com/polleyg/gcp-batch-ingestion-bigquery/blob/beam_sql/src/main/java/org/polleyg/BeamSQLMagic.java>
>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> *Until Next Time!*
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Rose Thị Nguyễn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Rose Thị Nguyễn
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Rose Thị Nguyễn
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> *Aizhamal Nurmamat kyzy*
>>>>>> Open Source Program Manager
>>>>>> 646-355-9740 <(646)%20355-9740> Mobile
>>>>>> 601 North 34th Street, Seattle, WA 98103
>>>>>> <https://maps.google.com/?q=601+North+34th+Street,+Seattle,+WA+98103&entry=gmail&source=g>
>>>>>>
>>>>>>
>>>
>>
>> --
>> Rose Thị Nguyễn
>>
>
Re: Apache Beam Newsletter - February/March 2019
Posted by Thomas Weise <th...@apache.org>.
The release blog is already on the radar for improvement [1].
I don't think there is a need to separate out the release blogs. That's if
they provide content that is valuable to users (IMO that's not exactly the
case today).
If release blogs are just reformatted JIRA reports, then maybe we can skip
them altogether (release notes are already linked from download page). In
that case I would much rather like to see the original JIRA report cleaned
up as part of the release process.
On the other hand, if release blogs are assembled similar to the newsletter
that we discuss here, through broader contributor input and with the aim to
provide more meaningful content to users, then I don't see why we need to
put them into a different area.
Overall, given proposed newsletter cadence of 2 months and existing release
cadence, we would probably still end up with a monthly piece of news.
Thanks,
Thomas
https://lists.apache.org/thread.html/224908775d8807eecc53e45f14f99d6a439beb8bb5165e3a813bf82d@%3Cdev.beam.apache.org%3E
On Tue, Mar 12, 2019 at 9:27 AM Rose Nguyen <rt...@google.com> wrote:
> Once every 2 months works. We include both big and small items. It'll
> provide the focus Thomas mentions but still catches the frequency that
> Pablo suggested for relevancy. To Alexey's point, it's difficult to
> navigate the more recent non-release blog posts (<1 year) because they are
> sprinkled in.
>
> A compromise for all of our points is to move the release notes to a
> separate section, and have a single section that's blog/news.
>
> And thanks for wanting to contribute, Aizhamal! Teaming up with you will
> make things run smoothly.
>
> On Tue, Mar 12, 2019 at 3:58 AM Alexey Romanenko <ar...@gmail.com>
> wrote:
>
>> +1 for “News” section. Though, I’d propose to exclude “release notes”
>> posts from “Blog” section list (since now we have quite regular releases,
>> it makes blog posts list not very readable for users) and move them to new
>> created News section. Also, this section could include the posts about
>> other Beam events, like meet-ups, conferences, project updates, etc. I’d
>> keep “Blog” section for more technical posts.
>>
>> On 12 Mar 2019, at 06:31, Pablo Estrada <pa...@google.com> wrote:
>>
>> I agree that the newsletter fits well as a blog post. I think that'd work
>> best.
>>
>> As for the cadence, I think quarterly would be a bit too infrequent. I
>> like once a month, or once every other month to have at least one per
>> release. Though happy to hear other people's thoughts.
>> Best
>> -P.
>>
>> On Mon, Mar 11, 2019 at 6:42 PM Thomas Weise <th...@apache.org> wrote:
>>
>>> +1 for single blog/news section
>>>
>>> Also I wouldn't mind quarterly cadence, to provide more focus for folks
>>> to contribute.
>>>
>>> On Mon, Mar 11, 2019 at 6:35 PM Kenneth Knowles <ke...@apache.org> wrote:
>>>
>>>> Could the newsletter be a blog entry? If you check out
>>>> https://blogs.apache.org/ many of the posts are "Apache News
>>>> Round-up". We could rename the "Blog" section to "News" if you ask me.
>>>>
>>>> Kenn
>>>>
>>>> On Mon, Mar 11, 2019 at 5:13 PM Aizhamal Nurmamat kyzy <
>>>> aizhamal@google.com> wrote:
>>>>
>>>>> Hello everyone,
>>>>>
>>>>> I had a chat with Rose on how I can support the effort and keep
>>>>> sending the newsletters on a monthly basis. The new workflow would look as
>>>>> follows:
>>>>>
>>>>> 1. Send out the same [CALL FOR ITEMS] where you can contribute to
>>>>> the Google doc with deadlines
>>>>> 2. Edit the doc after the deadline
>>>>> 3. Convert the file into Markdown
>>>>> 4. Send in PR to add the file to Beam repo in Newsletter directory
>>>>> 5. Have people send their fixes/updates through PRs
>>>>>
>>>>> In this effort, I can support Rose in steps 3 & 4.
>>>>>
>>>>> We would also need:
>>>>>
>>>>> - Create a Newsletter section under the Community tab
>>>>> - Write guidelines on newsletter contributions
>>>>> - Make a note about timing e.g. if upcoming event, then add to the
>>>>> next newsletter
>>>>>
>>>>> How does that sound to you all?
>>>>>
>>>>>
>>>>> On Wed, Mar 6, 2019 at 9:19 PM Rose Nguyen <rt...@google.com>
>>>>> wrote:
>>>>>
>>>>>> Good points. With the suggested workflow I think I can support a
>>>>>> quarterly newsletter. I'm also happy to get more involvement from others to
>>>>>> do this work and we can see what cadence that allows.
>>>>>>
>>>>>> On Wed, Mar 6, 2019 at 8:22 PM Kenneth Knowles <ke...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Good points Melissa & Austin.
>>>>>>>
>>>>>>> - archive in the repo & on the website
>>>>>>> - put missed items on the next newsletter, so anyone following sees
>>>>>>> them
>>>>>>>
>>>>>>> Kenn
>>>>>>>
>>>>>>> On Wed, Mar 6, 2019 at 3:26 PM Suneel Marthi <sm...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I believe there was also a Beam workshop or working session in
>>>>>>>> Warsaw last week.
>>>>>>>>
>>>>>>>> On Wed, Mar 6, 2019 at 6:20 PM Austin Bennett <
>>>>>>>> whatwouldaustindo@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> +1 for archive in our repo.
>>>>>>>>>
>>>>>>>>> I do follow the newsletter, but am unlikely to go back and look
>>>>>>>>> into the past for changes/updates.
>>>>>>>>>
>>>>>>>>> Would suggest that things that get missed in one newsletter (a
>>>>>>>>> concrete example, Suneel's talks not mentioned in the newsletter) would get
>>>>>>>>> published in the next iteration, rather than editing the past 'published'
>>>>>>>>> newsletter. Put another way, save editing the past for corrections (typos,
>>>>>>>>> things being incorrect). Else, I imagine that I'm unlikely to catch a
>>>>>>>>> great announcement that warranted being in the newsletter in the first
>>>>>>>>> place. This certainly works better with a regular/frequent release
>>>>>>>>> cadence, like we arrived at for version releases (then, if something misses
>>>>>>>>> one cut, it is not too big a deal, as the next release is coming soon).
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Mar 6, 2019 at 12:50 PM Melissa Pashniak <
>>>>>>>>> melissapa@google.com> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> For step #2 (publishing onto the website), I think it would be
>>>>>>>>>> good to stay consistent with our existing workflows if possible. Rather
>>>>>>>>>> than using an external tool, what about:
>>>>>>>>>>
>>>>>>>>>> After a google doc newsletter draft is ready, convert it into a
>>>>>>>>>> standard markdown file and put it into our GitHub repo, perhaps in a new
>>>>>>>>>> newsletter directory in the website community directory [1]. These would be
>>>>>>>>>> listed for browsing on a Newsletters page as mentioned in step #4. People
>>>>>>>>>> can then just open a PR to add missing things to the pages later, and the
>>>>>>>>>> newsletter will be automatically updated on the website through our
>>>>>>>>>> standard website workflow. It also avoids the potential issue of the source
>>>>>>>>>> google docs disappearing in the future, as they are stored in a community
>>>>>>>>>> location.
>>>>>>>>>>
>>>>>>>>>> [1]
>>>>>>>>>> https://github.com/apache/beam/tree/master/website/src/community
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Mar 6, 2019 at 10:36 AM Rose Nguyen <rt...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> I think that would be a great idea to change formats to help
>>>>>>>>>>> with distribution. I'm open to suggestions! I'm currently using a Google
>>>>>>>>>>> doc to collect and edit, then copy/paste sending the newsletter out
>>>>>>>>>>> directly, based on an interpretation of this discussion
>>>>>>>>>>> <https://lists.apache.org/thread.html/1f638eae43fe8abcb2f8752141c96d3dbdac86a583e0790044eea727@%3Cdev.beam.apache.org%3E>
>>>>>>>>>>> .
>>>>>>>>>>>
>>>>>>>>>>> How about this doc->website->Beam site workflow?:
>>>>>>>>>>>
>>>>>>>>>>> 1. The same usual newsletter [CALL FOR ITEMS] where you can
>>>>>>>>>>> contribute to the google doc, with soft deadlines for when I'll publish.
>>>>>>>>>>> 2. I'll publish the doc itself onto a website.
>>>>>>>>>>> 3. The newsletter is mailed out in the same way, but now
>>>>>>>>>>> with a shareable website link.
>>>>>>>>>>> 4. We'll keep an index of archived newsletter web pages on
>>>>>>>>>>> the Beam site, under the Community tab.
>>>>>>>>>>> 5. If you want to submit more content after the soft
>>>>>>>>>>> deadline, add it to the google doc and let me know to republish. I don't
>>>>>>>>>>> want to make the publication changes automatic because that leaves us open
>>>>>>>>>>> to tampering.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> This process is more laggy, so I'd suggest doing a 2 month vs
>>>>>>>>>>> monthly newsletter cadence. If we're happy with this idea, I'll send in a
>>>>>>>>>>> website PR for a new "Newsletter" left nav item under Community.
>>>>>>>>>>>
>>>>>>>>>>> Here's an example of a published newsletter: Apache Beam
>>>>>>>>>>> February-March 2019
>>>>>>>>>>> <https://gdoc.pub/doc/e/2PACX-1vTQIS4WkxV-HpgX5Lb6q05g4-wuIVcYd82123Mp4Y6q9fMv6Ynwd-l7dI4TrMyCrKilyU-YsoitbnZB>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> - This link is permanent unless the principal google doc is
>>>>>>>>>>> deleted.
>>>>>>>>>>> - Changes to the google doc after web publication are not
>>>>>>>>>>> automatically published on the website to protect the information integrity.
>>>>>>>>>>> - Republishing is quick and easy for me if you let me know
>>>>>>>>>>> you've added more.
>>>>>>>>>>> - I'll improve the formatting later if we go with this route.
>>>>>>>>>>>
>>>>>>>>>>> Any thoughts?
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Mar 6, 2019 at 6:13 AM Thomas Weise <th...@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Similar to blog posts. A link that can be shared would also
>>>>>>>>>>>> help to distribute over other channels, such as Twitter.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Mar 6, 2019, 6:06 AM Ismaël Mejía <ie...@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> We should have these newsletters published somewhere with a
>>>>>>>>>>>>> fixed URL so we can add missing updates, I have at least missed putting
>>>>>>>>>>>>> stuff in the last 3 ones just to realize when the newsletter has been
>>>>>>>>>>>>> already 'published' (and sadly interesting features like zstd have not been
>>>>>>>>>>>>> announced because of this).
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Mar 6, 2019 at 2:48 PM Etienne Chauchot <
>>>>>>>>>>>>> echauchot@apache.org> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>> I would add in what's been done:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Work on cassandraIO (Etienne Chauchot, Mathieu Blanchard,
>>>>>>>>>>>>>> Frank Shahar) : refactorings, bugfixes, new where clause, security fix
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Etienne
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Le lundi 04 mars 2019 à 18:36 +0100, Suneel Marthi a écrit :
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Is this the final draft? - we had 2 beam talks at Big Data
>>>>>>>>>>>>>> Tech Warsaw last Wednesday - I can send the updates offline.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Mar 4, 2019 at 6:16 PM Rose Nguyen <
>>>>>>>>>>>>>> rtnguyen@google.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [image: Beam.png]
>>>>>>>>>>>>>> February-March 2019 | Newsletter
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> What’s been done
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Apache Beam 2.10.0 released (by: many contributors)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Download the release here.
>>>>>>>>>>>>>> <https://beam.apache.org/get-started/downloads/>
>>>>>>>>>>>>>> - See the blog post
>>>>>>>>>>>>>> <https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html>
>>>>>>>>>>>>>> for more details.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Apache Beam awarded the 2019 Technology of the Year Award!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - InfoWorld just awarded Beam the 2019 Technology of the
>>>>>>>>>>>>>> Year Award.
>>>>>>>>>>>>>> - See this article
>>>>>>>>>>>>>> <https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html?nsdr=true>
>>>>>>>>>>>>>> for more details.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Kettle Beam 0.5 released with support for flink (by: Matt
>>>>>>>>>>>>>> Casters)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Kettle now supports Apache Flink as well as Cloud
>>>>>>>>>>>>>> Dataflow and Spark.
>>>>>>>>>>>>>> - See Matt’s Blog
>>>>>>>>>>>>>> <http://sandbox.kettle.be/wordpress/index.php/2019/02/24/kettle-beam-update-0-5-0/>
>>>>>>>>>>>>>> for more details.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> What we’re working on...
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Apache Beam 2.11.0 release (by: many contributors)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hive Metastore Table provider for SQL (by: Anton Kedin)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Support for plugging table providers through Beam SQL
>>>>>>>>>>>>>> API to allow obtaining table schemas from external sources.
>>>>>>>>>>>>>> - See the PR <https://github.com/apache/beam/pull/7746>
>>>>>>>>>>>>>> for more details.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> User Defined Coders for the Beam Go SDK (by: Robert Burke)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Working on expanding the variety of user defined types
>>>>>>>>>>>>>> that can be a member of a PCollection in the Go SDK.
>>>>>>>>>>>>>> - See BEAM-3306
>>>>>>>>>>>>>> <https://issues.apache.org/jira/browse/BEAM-3306> for
>>>>>>>>>>>>>> more details.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen,
>>>>>>>>>>>>>> Mark Liu, Robbe Sneyders, Juta Staes, Valentyn Tymofieiev)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Beam 2.11.0 is the first release offering partial
>>>>>>>>>>>>>> Python 3 support.
>>>>>>>>>>>>>> - Many thanks to all contributors who helped to reach
>>>>>>>>>>>>>> this milestone.
>>>>>>>>>>>>>> - IO availablility on Python 3 is currently limited and
>>>>>>>>>>>>>> only Python 3.5 version has been tested extensively.
>>>>>>>>>>>>>> - Stay tuned on BEAM-1251 for more details.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Notebooks for quickstarts and custom I/O (by: David Cavazos)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Adding IPython notebooks and snippets
>>>>>>>>>>>>>> - See [BEAM-6557]
>>>>>>>>>>>>>> <https://github.com/apache/beam/pull/7679> for more
>>>>>>>>>>>>>> details.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> New members
>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> New PMC member!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Etienne Chauchot, Nantes, France
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> New Committers!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Gleb Kanterov, Stockholm, Sweden
>>>>>>>>>>>>>> - Michael Luckey
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> New Contributors!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Kyle Weaver, San Francisco, CA
>>>>>>>>>>>>>> - Would like to help begin implementing portability
>>>>>>>>>>>>>> support for the Spark runner
>>>>>>>>>>>>>> - Tanay Tummapalli, Delhi, India
>>>>>>>>>>>>>> - Would like to contribute to Open Source this summer as
>>>>>>>>>>>>>> part of Google Summer of Code
>>>>>>>>>>>>>> - Brian Hulette, Seattle, WA
>>>>>>>>>>>>>> - Contributing to Beam Portability
>>>>>>>>>>>>>> - Michał Walenia, Warsaw, Poland
>>>>>>>>>>>>>> - Working on integration and load testing
>>>>>>>>>>>>>> - Daniel Chen, San Francisco, CA
>>>>>>>>>>>>>> - Working on Beam Samza runner
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Talks & meetups
>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Plugin Machine Intelligence and Apache Beam with Pentaho -
>>>>>>>>>>>>>> Feb 7 @ London
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Watch the How to Run Kettle on Apache Beam video here
>>>>>>>>>>>>>> <https://skillsmatter.com/skillscasts/13405-how-to-run-kettle-on-apache-beam#video>.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - See event details here
>>>>>>>>>>>>>> <https://www.meetup.com/Pentaho-London-User-Group/events/256773962/>
>>>>>>>>>>>>>> ..
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @
>>>>>>>>>>>>>> San Francisco, CA
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Organized by Thomas Weise and Austin Bennet, with
>>>>>>>>>>>>>> speakers Tyler Akidau, Robert Crowe, Thomas Weise and Amar Pai
>>>>>>>>>>>>>> - See event details here
>>>>>>>>>>>>>> <https://www.meetup.com/San-Francisco-Apache-Beam/events/257482350/>
>>>>>>>>>>>>>> and the slides for these presentation: Overview of Apache
>>>>>>>>>>>>>> Beam and TensorFlow Transform (TFX) with Apache Beam
>>>>>>>>>>>>>> <http://s.apache.org/beam-intro-feb-2019>, Python
>>>>>>>>>>>>>> Streaming Pipelines with Beam on Flink
>>>>>>>>>>>>>> <http://go.lyft.com/python-flink-beam-meetup-2019>, Dynamic
>>>>>>>>>>>>>> pricing of Lyft rides using streaming
>>>>>>>>>>>>>> <https://www.slideshare.net/AmarPai2/dynamic-pricing-of-lyft-rides-using-streaming>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> .
>>>>>>>>>>>>>> Flink meetup - Feb 21@ Seattle, WA
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Speakers from Alibaba, Google, and Uber gave talks
>>>>>>>>>>>>>> about Apache Flink with Hive, Tensorflow, Beam, and AthenaX.
>>>>>>>>>>>>>> - See event details here
>>>>>>>>>>>>>> <https://www.meetup.com/seattle-flink/events/258723322/>
>>>>>>>>>>>>>> and presentations here
>>>>>>>>>>>>>> <https://www.slideshare.net/BowenLi9/presentations>.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Beam Summit Europe 2019 - June 19-20 @ Berlin
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Beam Summit Europe 2019 will take place in Berlin on
>>>>>>>>>>>>>> June 19-20.
>>>>>>>>>>>>>> - Speaker CfP and other details to follow soon!
>>>>>>>>>>>>>> - Twitter announcement!
>>>>>>>>>>>>>> <https://twitter.com/matthiasbaetens/status/1098854758893273088>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Resources
>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Apache Jira Beginner’s Guide (by: Daniel Oliveira)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - A guide
>>>>>>>>>>>>>> <https://cwiki.apache.org/confluence/display/BEAM/Beam+Jira+Beginner%27s+Guide>
>>>>>>>>>>>>>> to introduce Beam contributors to the basics of using the Apache Jira for
>>>>>>>>>>>>>> Beam development. Feedback welcomed!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> An approach to community building from Apache Beam (by: Kenn
>>>>>>>>>>>>>> Knowles)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - The Apache Software Foundation has published committer
>>>>>>>>>>>>>> guidelines to help Beam's community building work.
>>>>>>>>>>>>>> - See the post
>>>>>>>>>>>>>> <https://blogs.apache.org/comdev/date/20190222> on the
>>>>>>>>>>>>>> ASF blog.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Exploring Beam SQL on Google Cloud Platform (by: Graham
>>>>>>>>>>>>>> Polley)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - “In this article, I’ll dive into this new feature of
>>>>>>>>>>>>>> Beam, and see how it works by using a pipeline to read a data file from
>>>>>>>>>>>>>> GCS, transform it, and then perform a basic calculation on the values
>>>>>>>>>>>>>> contained in the file”.
>>>>>>>>>>>>>> - See article
>>>>>>>>>>>>>> <https://medium.com/weareservian/exploring-beam-sql-on-google-cloud-platform-b6c77f9b4af4>
>>>>>>>>>>>>>> and full source code
>>>>>>>>>>>>>> <https://github.com/polleyg/gcp-batch-ingestion-bigquery/blob/beam_sql/src/main/java/org/polleyg/BeamSQLMagic.java>
>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *Until Next Time!*
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Rose Thị Nguyễn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Rose Thị Nguyễn
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Rose Thị Nguyễn
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> *Aizhamal Nurmamat kyzy*
>>>>> Open Source Program Manager
>>>>> 646-355-9740 <(646)%20355-9740> Mobile
>>>>> 601 North 34th Street, Seattle, WA 98103
>>>>> <https://maps.google.com/?q=601+North+34th+Street,+Seattle,+WA+98103&entry=gmail&source=g>
>>>>>
>>>>>
>>
>
> --
> Rose Thị Nguyễn
>
Re: Apache Beam Newsletter - February/March 2019
Posted by Rose Nguyen <rt...@google.com>.
Once every 2 months works. We include both big and small items. It'll
provide the focus Thomas mentions but still catches the frequency that
Pablo suggested for relevancy. To Alexey's point, it's difficult to
navigate the more recent non-release blog posts (<1 year) because they are
sprinkled in.
A compromise for all of our points is to move the release notes to a
separate section, and have a single section that's blog/news.
And thanks for wanting to contribute, Aizhamal! Teaming up with you will
make things run smoothly.
On Tue, Mar 12, 2019 at 3:58 AM Alexey Romanenko <ar...@gmail.com>
wrote:
> +1 for “News” section. Though, I’d propose to exclude “release notes”
> posts from “Blog” section list (since now we have quite regular releases,
> it makes blog posts list not very readable for users) and move them to new
> created News section. Also, this section could include the posts about
> other Beam events, like meet-ups, conferences, project updates, etc. I’d
> keep “Blog” section for more technical posts.
>
> On 12 Mar 2019, at 06:31, Pablo Estrada <pa...@google.com> wrote:
>
> I agree that the newsletter fits well as a blog post. I think that'd work
> best.
>
> As for the cadence, I think quarterly would be a bit too infrequent. I
> like once a month, or once every other month to have at least one per
> release. Though happy to hear other people's thoughts.
> Best
> -P.
>
> On Mon, Mar 11, 2019 at 6:42 PM Thomas Weise <th...@apache.org> wrote:
>
>> +1 for single blog/news section
>>
>> Also I wouldn't mind quarterly cadence, to provide more focus for folks
>> to contribute.
>>
>> On Mon, Mar 11, 2019 at 6:35 PM Kenneth Knowles <ke...@apache.org> wrote:
>>
>>> Could the newsletter be a blog entry? If you check out
>>> https://blogs.apache.org/ many of the posts are "Apache News Round-up".
>>> We could rename the "Blog" section to "News" if you ask me.
>>>
>>> Kenn
>>>
>>> On Mon, Mar 11, 2019 at 5:13 PM Aizhamal Nurmamat kyzy <
>>> aizhamal@google.com> wrote:
>>>
>>>> Hello everyone,
>>>>
>>>> I had a chat with Rose on how I can support the effort and keep sending
>>>> the newsletters on a monthly basis. The new workflow would look as follows:
>>>>
>>>> 1. Send out the same [CALL FOR ITEMS] where you can contribute to
>>>> the Google doc with deadlines
>>>> 2. Edit the doc after the deadline
>>>> 3. Convert the file into Markdown
>>>> 4. Send in PR to add the file to Beam repo in Newsletter directory
>>>> 5. Have people send their fixes/updates through PRs
>>>>
>>>> In this effort, I can support Rose in steps 3 & 4.
>>>>
>>>> We would also need:
>>>>
>>>> - Create a Newsletter section under the Community tab
>>>> - Write guidelines on newsletter contributions
>>>> - Make a note about timing e.g. if upcoming event, then add to the
>>>> next newsletter
>>>>
>>>> How does that sound to you all?
>>>>
>>>>
>>>> On Wed, Mar 6, 2019 at 9:19 PM Rose Nguyen <rt...@google.com> wrote:
>>>>
>>>>> Good points. With the suggested workflow I think I can support a
>>>>> quarterly newsletter. I'm also happy to get more involvement from others to
>>>>> do this work and we can see what cadence that allows.
>>>>>
>>>>> On Wed, Mar 6, 2019 at 8:22 PM Kenneth Knowles <ke...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Good points Melissa & Austin.
>>>>>>
>>>>>> - archive in the repo & on the website
>>>>>> - put missed items on the next newsletter, so anyone following sees
>>>>>> them
>>>>>>
>>>>>> Kenn
>>>>>>
>>>>>> On Wed, Mar 6, 2019 at 3:26 PM Suneel Marthi <sm...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> I believe there was also a Beam workshop or working session in
>>>>>>> Warsaw last week.
>>>>>>>
>>>>>>> On Wed, Mar 6, 2019 at 6:20 PM Austin Bennett <
>>>>>>> whatwouldaustindo@gmail.com> wrote:
>>>>>>>
>>>>>>>> +1 for archive in our repo.
>>>>>>>>
>>>>>>>> I do follow the newsletter, but am unlikely to go back and look
>>>>>>>> into the past for changes/updates.
>>>>>>>>
>>>>>>>> Would suggest that things that get missed in one newsletter (a
>>>>>>>> concrete example, Suneel's talks not mentioned in the newsletter) would get
>>>>>>>> published in the next iteration, rather than editing the past 'published'
>>>>>>>> newsletter. Put another way, save editing the past for corrections (typos,
>>>>>>>> things being incorrect). Else, I imagine that I'm unlikely to catch a
>>>>>>>> great announcement that warranted being in the newsletter in the first
>>>>>>>> place. This certainly works better with a regular/frequent release
>>>>>>>> cadence, like we arrived at for version releases (then, if something misses
>>>>>>>> one cut, it is not too big a deal, as the next release is coming soon).
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Mar 6, 2019 at 12:50 PM Melissa Pashniak <
>>>>>>>> melissapa@google.com> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> For step #2 (publishing onto the website), I think it would be
>>>>>>>>> good to stay consistent with our existing workflows if possible. Rather
>>>>>>>>> than using an external tool, what about:
>>>>>>>>>
>>>>>>>>> After a google doc newsletter draft is ready, convert it into a
>>>>>>>>> standard markdown file and put it into our GitHub repo, perhaps in a new
>>>>>>>>> newsletter directory in the website community directory [1]. These would be
>>>>>>>>> listed for browsing on a Newsletters page as mentioned in step #4. People
>>>>>>>>> can then just open a PR to add missing things to the pages later, and the
>>>>>>>>> newsletter will be automatically updated on the website through our
>>>>>>>>> standard website workflow. It also avoids the potential issue of the source
>>>>>>>>> google docs disappearing in the future, as they are stored in a community
>>>>>>>>> location.
>>>>>>>>>
>>>>>>>>> [1]
>>>>>>>>> https://github.com/apache/beam/tree/master/website/src/community
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Mar 6, 2019 at 10:36 AM Rose Nguyen <rt...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I think that would be a great idea to change formats to help with
>>>>>>>>>> distribution. I'm open to suggestions! I'm currently using a Google doc to
>>>>>>>>>> collect and edit, then copy/paste sending the newsletter out directly, based
>>>>>>>>>> on an interpretation of this discussion
>>>>>>>>>> <https://lists.apache.org/thread.html/1f638eae43fe8abcb2f8752141c96d3dbdac86a583e0790044eea727@%3Cdev.beam.apache.org%3E>
>>>>>>>>>> .
>>>>>>>>>>
>>>>>>>>>> How about this doc->website->Beam site workflow?:
>>>>>>>>>>
>>>>>>>>>> 1. The same usual newsletter [CALL FOR ITEMS] where you can
>>>>>>>>>> contribute to the google doc, with soft deadlines for when I'll publish.
>>>>>>>>>> 2. I'll publish the doc itself onto a website.
>>>>>>>>>> 3. The newsletter is mailed out in the same way, but now with
>>>>>>>>>> a shareable website link.
>>>>>>>>>> 4. We'll keep an index of archived newsletter web pages on
>>>>>>>>>> the Beam site, under the Community tab.
>>>>>>>>>> 5. If you want to submit more content after the soft
>>>>>>>>>> deadline, add it to the google doc and let me know to republish. I don't
>>>>>>>>>> want to make the publication changes automatic because that leaves us open
>>>>>>>>>> to tampering.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> This process is more laggy, so I'd suggest doing a 2 month vs
>>>>>>>>>> monthly newsletter cadence. If we're happy with this idea, I'll send in a
>>>>>>>>>> website PR for a new "Newsletter" left nav item under Community.
>>>>>>>>>>
>>>>>>>>>> Here's an example of a published newsletter: Apache Beam
>>>>>>>>>> February-March 2019
>>>>>>>>>> <https://gdoc.pub/doc/e/2PACX-1vTQIS4WkxV-HpgX5Lb6q05g4-wuIVcYd82123Mp4Y6q9fMv6Ynwd-l7dI4TrMyCrKilyU-YsoitbnZB>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> - This link is permanent unless the principal google doc is
>>>>>>>>>> deleted.
>>>>>>>>>> - Changes to the google doc after web publication are not
>>>>>>>>>> automatically published on the website to protect the information integrity.
>>>>>>>>>> - Republishing is quick and easy for me if you let me know
>>>>>>>>>> you've added more.
>>>>>>>>>> - I'll improve the formatting later if we go with this route.
>>>>>>>>>>
>>>>>>>>>> Any thoughts?
>>>>>>>>>>
>>>>>>>>>> On Wed, Mar 6, 2019 at 6:13 AM Thomas Weise <th...@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Similar to blog posts. A link that can be shared would also help
>>>>>>>>>>> to distribute over other channels, such as Twitter.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Mar 6, 2019, 6:06 AM Ismaël Mejía <ie...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> We should have these newsletters published somewhere with a
>>>>>>>>>>>> fixed URL so we can add missing updates, I have at least missed putting
>>>>>>>>>>>> stuff in the last 3 ones just to realize when the newsletter has been
>>>>>>>>>>>> already 'published' (and sadly interesting features like zstd have not been
>>>>>>>>>>>> announced because of this).
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Mar 6, 2019 at 2:48 PM Etienne Chauchot <
>>>>>>>>>>>> echauchot@apache.org> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>> I would add in what's been done:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Work on cassandraIO (Etienne Chauchot, Mathieu Blanchard,
>>>>>>>>>>>>> Frank Shahar) : refactorings, bugfixes, new where clause, security fix
>>>>>>>>>>>>>
>>>>>>>>>>>>> Etienne
>>>>>>>>>>>>>
>>>>>>>>>>>>> Le lundi 04 mars 2019 à 18:36 +0100, Suneel Marthi a écrit :
>>>>>>>>>>>>>
>>>>>>>>>>>>> Is this the final draft? - we had 2 beam talks at Big Data
>>>>>>>>>>>>> Tech Warsaw last Wednesday - I can send the updates offline.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Mar 4, 2019 at 6:16 PM Rose Nguyen <
>>>>>>>>>>>>> rtnguyen@google.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> [image: Beam.png]
>>>>>>>>>>>>> February-March 2019 | Newsletter
>>>>>>>>>>>>>
>>>>>>>>>>>>> What’s been done
>>>>>>>>>>>>>
>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>
>>>>>>>>>>>>> Apache Beam 2.10.0 released (by: many contributors)
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Download the release here.
>>>>>>>>>>>>> <https://beam.apache.org/get-started/downloads/>
>>>>>>>>>>>>> - See the blog post
>>>>>>>>>>>>> <https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html>
>>>>>>>>>>>>> for more details.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Apache Beam awarded the 2019 Technology of the Year Award!
>>>>>>>>>>>>>
>>>>>>>>>>>>> - InfoWorld just awarded Beam the 2019 Technology of the
>>>>>>>>>>>>> Year Award.
>>>>>>>>>>>>> - See this article
>>>>>>>>>>>>> <https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html?nsdr=true>
>>>>>>>>>>>>> for more details.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Kettle Beam 0.5 released with support for flink (by: Matt
>>>>>>>>>>>>> Casters)
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Kettle now supports Apache Flink as well as Cloud
>>>>>>>>>>>>> Dataflow and Spark.
>>>>>>>>>>>>> - See Matt’s Blog
>>>>>>>>>>>>> <http://sandbox.kettle.be/wordpress/index.php/2019/02/24/kettle-beam-update-0-5-0/>
>>>>>>>>>>>>> for more details.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> What we’re working on...
>>>>>>>>>>>>>
>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>
>>>>>>>>>>>>> Apache Beam 2.11.0 release (by: many contributors)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hive Metastore Table provider for SQL (by: Anton Kedin)
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Support for plugging table providers through Beam SQL
>>>>>>>>>>>>> API to allow obtaining table schemas from external sources.
>>>>>>>>>>>>> - See the PR <https://github.com/apache/beam/pull/7746>
>>>>>>>>>>>>> for more details.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> User Defined Coders for the Beam Go SDK (by: Robert Burke)
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Working on expanding the variety of user defined types
>>>>>>>>>>>>> that can be a member of a PCollection in the Go SDK.
>>>>>>>>>>>>> - See BEAM-3306
>>>>>>>>>>>>> <https://issues.apache.org/jira/browse/BEAM-3306> for more
>>>>>>>>>>>>> details.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen,
>>>>>>>>>>>>> Mark Liu, Robbe Sneyders, Juta Staes, Valentyn Tymofieiev)
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Beam 2.11.0 is the first release offering partial Python
>>>>>>>>>>>>> 3 support.
>>>>>>>>>>>>> - Many thanks to all contributors who helped to reach this
>>>>>>>>>>>>> milestone.
>>>>>>>>>>>>> - IO availablility on Python 3 is currently limited and
>>>>>>>>>>>>> only Python 3.5 version has been tested extensively.
>>>>>>>>>>>>> - Stay tuned on BEAM-1251 for more details.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Notebooks for quickstarts and custom I/O (by: David Cavazos)
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Adding IPython notebooks and snippets
>>>>>>>>>>>>> - See [BEAM-6557]
>>>>>>>>>>>>> <https://github.com/apache/beam/pull/7679> for more
>>>>>>>>>>>>> details.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> New members
>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>
>>>>>>>>>>>>> New PMC member!
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Etienne Chauchot, Nantes, France
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> New Committers!
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Gleb Kanterov, Stockholm, Sweden
>>>>>>>>>>>>> - Michael Luckey
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> New Contributors!
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Kyle Weaver, San Francisco, CA
>>>>>>>>>>>>> - Would like to help begin implementing portability
>>>>>>>>>>>>> support for the Spark runner
>>>>>>>>>>>>> - Tanay Tummapalli, Delhi, India
>>>>>>>>>>>>> - Would like to contribute to Open Source this summer as
>>>>>>>>>>>>> part of Google Summer of Code
>>>>>>>>>>>>> - Brian Hulette, Seattle, WA
>>>>>>>>>>>>> - Contributing to Beam Portability
>>>>>>>>>>>>> - Michał Walenia, Warsaw, Poland
>>>>>>>>>>>>> - Working on integration and load testing
>>>>>>>>>>>>> - Daniel Chen, San Francisco, CA
>>>>>>>>>>>>> - Working on Beam Samza runner
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Talks & meetups
>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Plugin Machine Intelligence and Apache Beam with Pentaho - Feb
>>>>>>>>>>>>> 7 @ London
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Watch the How to Run Kettle on Apache Beam video here
>>>>>>>>>>>>> <https://skillsmatter.com/skillscasts/13405-how-to-run-kettle-on-apache-beam#video>.
>>>>>>>>>>>>>
>>>>>>>>>>>>> - See event details here
>>>>>>>>>>>>> <https://www.meetup.com/Pentaho-London-User-Group/events/256773962/>
>>>>>>>>>>>>> ..
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @ San
>>>>>>>>>>>>> Francisco, CA
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Organized by Thomas Weise and Austin Bennet, with
>>>>>>>>>>>>> speakers Tyler Akidau, Robert Crowe, Thomas Weise and Amar Pai
>>>>>>>>>>>>> - See event details here
>>>>>>>>>>>>> <https://www.meetup.com/San-Francisco-Apache-Beam/events/257482350/>
>>>>>>>>>>>>> and the slides for these presentation: Overview of Apache
>>>>>>>>>>>>> Beam and TensorFlow Transform (TFX) with Apache Beam
>>>>>>>>>>>>> <http://s.apache.org/beam-intro-feb-2019>, Python
>>>>>>>>>>>>> Streaming Pipelines with Beam on Flink
>>>>>>>>>>>>> <http://go.lyft.com/python-flink-beam-meetup-2019>, Dynamic
>>>>>>>>>>>>> pricing of Lyft rides using streaming
>>>>>>>>>>>>> <https://www.slideshare.net/AmarPai2/dynamic-pricing-of-lyft-rides-using-streaming>
>>>>>>>>>>>>>
>>>>>>>>>>>>> .
>>>>>>>>>>>>> Flink meetup - Feb 21@ Seattle, WA
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Speakers from Alibaba, Google, and Uber gave talks about
>>>>>>>>>>>>> Apache Flink with Hive, Tensorflow, Beam, and AthenaX.
>>>>>>>>>>>>> - See event details here
>>>>>>>>>>>>> <https://www.meetup.com/seattle-flink/events/258723322/>
>>>>>>>>>>>>> and presentations here
>>>>>>>>>>>>> <https://www.slideshare.net/BowenLi9/presentations>.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Beam Summit Europe 2019 - June 19-20 @ Berlin
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Beam Summit Europe 2019 will take place in Berlin on
>>>>>>>>>>>>> June 19-20.
>>>>>>>>>>>>> - Speaker CfP and other details to follow soon!
>>>>>>>>>>>>> - Twitter announcement!
>>>>>>>>>>>>> <https://twitter.com/matthiasbaetens/status/1098854758893273088>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Resources
>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>
>>>>>>>>>>>>> Apache Jira Beginner’s Guide (by: Daniel Oliveira)
>>>>>>>>>>>>>
>>>>>>>>>>>>> - A guide
>>>>>>>>>>>>> <https://cwiki.apache.org/confluence/display/BEAM/Beam+Jira+Beginner%27s+Guide>
>>>>>>>>>>>>> to introduce Beam contributors to the basics of using the Apache Jira for
>>>>>>>>>>>>> Beam development. Feedback welcomed!
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> An approach to community building from Apache Beam (by: Kenn
>>>>>>>>>>>>> Knowles)
>>>>>>>>>>>>>
>>>>>>>>>>>>> - The Apache Software Foundation has published committer
>>>>>>>>>>>>> guidelines to help Beam's community building work.
>>>>>>>>>>>>> - See the post
>>>>>>>>>>>>> <https://blogs.apache.org/comdev/date/20190222> on the ASF
>>>>>>>>>>>>> blog.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Exploring Beam SQL on Google Cloud Platform (by: Graham
>>>>>>>>>>>>> Polley)
>>>>>>>>>>>>>
>>>>>>>>>>>>> - “In this article, I’ll dive into this new feature of
>>>>>>>>>>>>> Beam, and see how it works by using a pipeline to read a data file from
>>>>>>>>>>>>> GCS, transform it, and then perform a basic calculation on the values
>>>>>>>>>>>>> contained in the file”.
>>>>>>>>>>>>> - See article
>>>>>>>>>>>>> <https://medium.com/weareservian/exploring-beam-sql-on-google-cloud-platform-b6c77f9b4af4>
>>>>>>>>>>>>> and full source code
>>>>>>>>>>>>> <https://github.com/polleyg/gcp-batch-ingestion-bigquery/blob/beam_sql/src/main/java/org/polleyg/BeamSQLMagic.java>
>>>>>>>>>>>>> .
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> *Until Next Time!*
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Rose Thị Nguyễn
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Rose Thị Nguyễn
>>>>>>>>>>
>>>>>>>>>
>>>>>
>>>>> --
>>>>> Rose Thị Nguyễn
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> *Aizhamal Nurmamat kyzy*
>>>> Open Source Program Manager
>>>> 646-355-9740 <(646)%20355-9740> Mobile
>>>> 601 North 34th Street, Seattle, WA 98103
>>>> <https://maps.google.com/?q=601+North+34th+Street,+Seattle,+WA+98103&entry=gmail&source=g>
>>>>
>>>>
>
--
Rose Thị Nguyễn
Re: Apache Beam Newsletter - February/March 2019
Posted by Alexey Romanenko <ar...@gmail.com>.
+1 for “News” section. Though, I’d propose to exclude “release notes” posts from “Blog” section list (since now we have quite regular releases, it makes blog posts list not very readable for users) and move them to new created News section. Also, this section could include the posts about other Beam events, like meet-ups, conferences, project updates, etc. I’d keep “Blog” section for more technical posts.
> On 12 Mar 2019, at 06:31, Pablo Estrada <pa...@google.com> wrote:
>
> I agree that the newsletter fits well as a blog post. I think that'd work best.
>
> As for the cadence, I think quarterly would be a bit too infrequent. I like once a month, or once every other month to have at least one per release. Though happy to hear other people's thoughts.
> Best
> -P.
>
> On Mon, Mar 11, 2019 at 6:42 PM Thomas Weise <thw@apache.org <ma...@apache.org>> wrote:
> +1 for single blog/news section
>
> Also I wouldn't mind quarterly cadence, to provide more focus for folks to contribute.
>
> On Mon, Mar 11, 2019 at 6:35 PM Kenneth Knowles <kenn@apache.org <ma...@apache.org>> wrote:
> Could the newsletter be a blog entry? If you check out https://blogs.apache.org/ <https://blogs.apache.org/> many of the posts are "Apache News Round-up". We could rename the "Blog" section to "News" if you ask me.
>
> Kenn
>
> On Mon, Mar 11, 2019 at 5:13 PM Aizhamal Nurmamat kyzy <aizhamal@google.com <ma...@google.com>> wrote:
> Hello everyone,
>
> I had a chat with Rose on how I can support the effort and keep sending the newsletters on a monthly basis. The new workflow would look as follows:
> Send out the same [CALL FOR ITEMS] where you can contribute to the Google doc with deadlines
> Edit the doc after the deadline
> Convert the file into Markdown
> Send in PR to add the file to Beam repo in Newsletter directory
> Have people send their fixes/updates through PRs
> In this effort, I can support Rose in steps 3 & 4.
>
> We would also need:
> Create a Newsletter section under the Community tab
> Write guidelines on newsletter contributions
> Make a note about timing e.g. if upcoming event, then add to the next newsletter
> How does that sound to you all?
>
>
> On Wed, Mar 6, 2019 at 9:19 PM Rose Nguyen <rtnguyen@google.com <ma...@google.com>> wrote:
> Good points. With the suggested workflow I think I can support a quarterly newsletter. I'm also happy to get more involvement from others to do this work and we can see what cadence that allows.
>
> On Wed, Mar 6, 2019 at 8:22 PM Kenneth Knowles <kenn@apache.org <ma...@apache.org>> wrote:
> Good points Melissa & Austin.
>
> - archive in the repo & on the website
> - put missed items on the next newsletter, so anyone following sees them
>
> Kenn
>
> On Wed, Mar 6, 2019 at 3:26 PM Suneel Marthi <smarthi@apache.org <ma...@apache.org>> wrote:
> I believe there was also a Beam workshop or working session in Warsaw last week.
>
> On Wed, Mar 6, 2019 at 6:20 PM Austin Bennett <whatwouldaustindo@gmail.com <ma...@gmail.com>> wrote:
> +1 for archive in our repo.
>
> I do follow the newsletter, but am unlikely to go back and look into the past for changes/updates.
>
> Would suggest that things that get missed in one newsletter (a concrete example, Suneel's talks not mentioned in the newsletter) would get published in the next iteration, rather than editing the past 'published' newsletter. Put another way, save editing the past for corrections (typos, things being incorrect). Else, I imagine that I'm unlikely to catch a great announcement that warranted being in the newsletter in the first place. This certainly works better with a regular/frequent release cadence, like we arrived at for version releases (then, if something misses one cut, it is not too big a deal, as the next release is coming soon).
>
>
>
>
> On Wed, Mar 6, 2019 at 12:50 PM Melissa Pashniak <melissapa@google.com <ma...@google.com>> wrote:
>
> For step #2 (publishing onto the website), I think it would be good to stay consistent with our existing workflows if possible. Rather than using an external tool, what about:
>
> After a google doc newsletter draft is ready, convert it into a standard markdown file and put it into our GitHub repo, perhaps in a new newsletter directory in the website community directory [1]. These would be listed for browsing on a Newsletters page as mentioned in step #4. People can then just open a PR to add missing things to the pages later, and the newsletter will be automatically updated on the website through our standard website workflow. It also avoids the potential issue of the source google docs disappearing in the future, as they are stored in a community location.
>
> [1] https://github.com/apache/beam/tree/master/website/src/community <https://github.com/apache/beam/tree/master/website/src/community>
>
>
> On Wed, Mar 6, 2019 at 10:36 AM Rose Nguyen <rtnguyen@google.com <ma...@google.com>> wrote:
> I think that would be a great idea to change formats to help with distribution. I'm open to suggestions! I'm currently using a Google doc to collect and edit, then copy/paste sending the newsletter out directly, based on an interpretation of this discussion <https://lists.apache.org/thread.html/1f638eae43fe8abcb2f8752141c96d3dbdac86a583e0790044eea727@%3Cdev.beam.apache.org%3E>.
>
> How about this doc->website->Beam site workflow?:
> The same usual newsletter [CALL FOR ITEMS] where you can contribute to the google doc, with soft deadlines for when I'll publish.
> I'll publish the doc itself onto a website.
> The newsletter is mailed out in the same way, but now with a shareable website link.
> We'll keep an index of archived newsletter web pages on the Beam site, under the Community tab.
> If you want to submit more content after the soft deadline, add it to the google doc and let me know to republish. I don't want to make the publication changes automatic because that leaves us open to tampering.
>
> This process is more laggy, so I'd suggest doing a 2 month vs monthly newsletter cadence. If we're happy with this idea, I'll send in a website PR for a new "Newsletter" left nav item under Community.
>
> Here's an example of a published newsletter: Apache Beam February-March 2019 <https://gdoc.pub/doc/e/2PACX-1vTQIS4WkxV-HpgX5Lb6q05g4-wuIVcYd82123Mp4Y6q9fMv6Ynwd-l7dI4TrMyCrKilyU-YsoitbnZB>
> This link is permanent unless the principal google doc is deleted.
> Changes to the google doc after web publication are not automatically published on the website to protect the information integrity.
> Republishing is quick and easy for me if you let me know you've added more.
> I'll improve the formatting later if we go with this route.
> Any thoughts?
>
> On Wed, Mar 6, 2019 at 6:13 AM Thomas Weise <thw@apache.org <ma...@apache.org>> wrote:
> Similar to blog posts. A link that can be shared would also help to distribute over other channels, such as Twitter.
>
>
> On Wed, Mar 6, 2019, 6:06 AM Ismaël Mejía <iemejia@gmail.com <ma...@gmail.com>> wrote:
> We should have these newsletters published somewhere with a fixed URL so we can add missing updates, I have at least missed putting stuff in the last 3 ones just to realize when the newsletter has been already 'published' (and sadly interesting features like zstd have not been announced because of this).
>
> On Wed, Mar 6, 2019 at 2:48 PM Etienne Chauchot <echauchot@apache.org <ma...@apache.org>> wrote:
> Hi,
> I would add in what's been done:
>
> Work on cassandraIO (Etienne Chauchot, Mathieu Blanchard, Frank Shahar) : refactorings, bugfixes, new where clause, security fix
>
> Etienne
>
> Le lundi 04 mars 2019 à 18:36 +0100, Suneel Marthi a écrit :
>> Is this the final draft? - we had 2 beam talks at Big Data Tech Warsaw last Wednesday - I can send the updates offline.
>>
>> On Mon, Mar 4, 2019 at 6:16 PM Rose Nguyen <rtnguyen@google.com <ma...@google.com>> wrote:
>>>
>>>
>>> February-March 2019 | Newsletter
>>>
>>> What’s been done
>>>
>>> Apache Beam 2.10.0 released (by: many contributors)
>>> Download the release here. <https://beam.apache.org/get-started/downloads/>
>>> See the blog post <https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html> for more details.
>>>
>>> Apache Beam awarded the 2019 Technology of the Year Award!
>>> InfoWorld just awarded Beam the 2019 Technology of the Year Award.
>>> See this article <https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html?nsdr=true> for more details.
>>>
>>> Kettle Beam 0.5 released with support for flink (by: Matt Casters)
>>> Kettle now supports Apache Flink as well as Cloud Dataflow and Spark.
>>> See Matt’s Blog <http://sandbox.kettle.be/wordpress/index.php/2019/02/24/kettle-beam-update-0-5-0/> for more details.
>>>
>>>
>>> What we’re working on...
>>>
>>> Apache Beam 2.11.0 release (by: many contributors)
>>>
>>>
>>> Hive Metastore Table provider for SQL (by: Anton Kedin)
>>> Support for plugging table providers through Beam SQL API to allow obtaining table schemas from external sources.
>>> See the PR <https://github.com/apache/beam/pull/7746> for more details.
>>>
>>> User Defined Coders for the Beam Go SDK (by: Robert Burke)
>>> Working on expanding the variety of user defined types that can be a member of a PCollection in the Go SDK.
>>> See BEAM-3306 <https://issues.apache.org/jira/browse/BEAM-3306> for more details.
>>>
>>> Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen, Mark Liu, Robbe Sneyders, Juta Staes, Valentyn Tymofieiev)
>>> Beam 2.11.0 is the first release offering partial Python 3 support.
>>> Many thanks to all contributors who helped to reach this milestone.
>>> IO availablility on Python 3 is currently limited and only Python 3.5 version has been tested extensively.
>>> Stay tuned on BEAM-1251 for more details.
>>>
>>> Notebooks for quickstarts and custom I/O (by: David Cavazos)
>>> Adding IPython notebooks and snippets
>>> See [BEAM-6557] <https://github.com/apache/beam/pull/7679> for more details.
>>>
>>>
>>>
>>> New members
>>>
>>> New PMC member!
>>> Etienne Chauchot, Nantes, France
>>>
>>> New Committers!
>>> Gleb Kanterov, Stockholm, Sweden
>>> Michael Luckey
>>>
>>> New Contributors!
>>> Kyle Weaver, San Francisco, CA
>>> Would like to help begin implementing portability support for the Spark runner
>>> Tanay Tummapalli, Delhi, India
>>> Would like to contribute to Open Source this summer as part of Google Summer of Code
>>> Brian Hulette, Seattle, WA
>>> Contributing to Beam Portability
>>> Michał Walenia, Warsaw, Poland
>>> Working on integration and load testing
>>> Daniel Chen, San Francisco, CA
>>> Working on Beam Samza runner
>>>
>>>
>>> Talks & meetups
>>>
>>>
>>> Plugin Machine Intelligence and Apache Beam with Pentaho - Feb 7 @ London
>>> Watch the How to Run Kettle on Apache Beam video here <https://skillsmatter.com/skillscasts/13405-how-to-run-kettle-on-apache-beam#video>.
>>> See event details here <https://www.meetup.com/Pentaho-London-User-Group/events/256773962/>..
>>>
>>> Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @ San Francisco, CA
>>> Organized by Thomas Weise and Austin Bennet, with speakers Tyler Akidau, Robert Crowe, Thomas Weise and Amar Pai
>>> See event details here <https://www.meetup.com/San-Francisco-Apache-Beam/events/257482350/> and the slides for these presentation: Overview of Apache Beam and TensorFlow Transform (TFX) with Apache Beam <http://s.apache.org/beam-intro-feb-2019>, Python Streaming Pipelines with Beam on Flink <http://go.lyft.com/python-flink-beam-meetup-2019>, Dynamic pricing of Lyft rides using streaming <https://www.slideshare.net/AmarPai2/dynamic-pricing-of-lyft-rides-using-streaming>
>>> .
>>> Flink meetup - Feb 21@ Seattle, WA
>>> Speakers from Alibaba, Google, and Uber gave talks about Apache Flink with Hive, Tensorflow, Beam, and AthenaX.
>>> See event details here <https://www.meetup.com/seattle-flink/events/258723322/> and presentations here <https://www.slideshare.net/BowenLi9/presentations>.
>>>
>>> Beam Summit Europe 2019 - June 19-20 @ Berlin
>>> Beam Summit Europe 2019 will take place in Berlin on June 19-20.
>>> Speaker CfP and other details to follow soon!
>>> Twitter announcement! <https://twitter.com/matthiasbaetens/status/1098854758893273088>
>>>
>>> Resources
>>>
>>> Apache Jira Beginner’s Guide (by: Daniel Oliveira)
>>> A guide <https://cwiki.apache.org/confluence/display/BEAM/Beam+Jira+Beginner%27s+Guide> to introduce Beam contributors to the basics of using the Apache Jira for Beam development. Feedback welcomed!
>>>
>>> An approach to community building from Apache Beam (by: Kenn Knowles)
>>> The Apache Software Foundation has published committer guidelines to help Beam's community building work.
>>> See the post <https://blogs.apache.org/comdev/date/20190222> on the ASF blog.
>>>
>>> Exploring Beam SQL on Google Cloud Platform (by: Graham Polley)
>>> “In this article, I’ll dive into this new feature of Beam, and see how it works by using a pipeline to read a data file from GCS, transform it, and then perform a basic calculation on the values contained in the file”.
>>> See article <https://medium.com/weareservian/exploring-beam-sql-on-google-cloud-platform-b6c77f9b4af4> and full source code <https://github.com/polleyg/gcp-batch-ingestion-bigquery/blob/beam_sql/src/main/java/org/polleyg/BeamSQLMagic.java>.
>>>
>>> Until Next Time!
>>>
>>> --
>>> Rose Thị Nguyễn
>
>
> --
> Rose Thị Nguyễn
>
>
> --
> Rose Thị Nguyễn
>
>
> --
>
> Aizhamal Nurmamat kyzy
> Open Source Program Manager
> 646-355-9740 <tel:(646)%20355-9740> Mobile
> 601 North 34th Street, Seattle, WA 98103 <https://maps.google.com/?q=601+North+34th+Street,+Seattle,+WA+98103&entry=gmail&source=g>
>
Re: Apache Beam Newsletter - February/March 2019
Posted by Pablo Estrada <pa...@google.com>.
I agree that the newsletter fits well as a blog post. I think that'd work
best.
As for the cadence, I think quarterly would be a bit too infrequent. I like
once a month, or once every other month to have at least one per release.
Though happy to hear other people's thoughts.
Best
-P.
On Mon, Mar 11, 2019 at 6:42 PM Thomas Weise <th...@apache.org> wrote:
> +1 for single blog/news section
>
> Also I wouldn't mind quarterly cadence, to provide more focus for folks to
> contribute.
>
> On Mon, Mar 11, 2019 at 6:35 PM Kenneth Knowles <ke...@apache.org> wrote:
>
>> Could the newsletter be a blog entry? If you check out
>> https://blogs.apache.org/ many of the posts are "Apache News Round-up".
>> We could rename the "Blog" section to "News" if you ask me.
>>
>> Kenn
>>
>> On Mon, Mar 11, 2019 at 5:13 PM Aizhamal Nurmamat kyzy <
>> aizhamal@google.com> wrote:
>>
>>> Hello everyone,
>>>
>>> I had a chat with Rose on how I can support the effort and keep sending
>>> the newsletters on a monthly basis. The new workflow would look as follows:
>>>
>>> 1. Send out the same [CALL FOR ITEMS] where you can contribute to
>>> the Google doc with deadlines
>>> 2. Edit the doc after the deadline
>>> 3. Convert the file into Markdown
>>> 4. Send in PR to add the file to Beam repo in Newsletter directory
>>> 5. Have people send their fixes/updates through PRs
>>>
>>> In this effort, I can support Rose in steps 3 & 4.
>>>
>>> We would also need:
>>>
>>> - Create a Newsletter section under the Community tab
>>> - Write guidelines on newsletter contributions
>>> - Make a note about timing e.g. if upcoming event, then add to the
>>> next newsletter
>>>
>>> How does that sound to you all?
>>>
>>>
>>> On Wed, Mar 6, 2019 at 9:19 PM Rose Nguyen <rt...@google.com> wrote:
>>>
>>>> Good points. With the suggested workflow I think I can support a
>>>> quarterly newsletter. I'm also happy to get more involvement from others to
>>>> do this work and we can see what cadence that allows.
>>>>
>>>> On Wed, Mar 6, 2019 at 8:22 PM Kenneth Knowles <ke...@apache.org> wrote:
>>>>
>>>>> Good points Melissa & Austin.
>>>>>
>>>>> - archive in the repo & on the website
>>>>> - put missed items on the next newsletter, so anyone following sees
>>>>> them
>>>>>
>>>>> Kenn
>>>>>
>>>>> On Wed, Mar 6, 2019 at 3:26 PM Suneel Marthi <sm...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> I believe there was also a Beam workshop or working session in Warsaw
>>>>>> last week.
>>>>>>
>>>>>> On Wed, Mar 6, 2019 at 6:20 PM Austin Bennett <
>>>>>> whatwouldaustindo@gmail.com> wrote:
>>>>>>
>>>>>>> +1 for archive in our repo.
>>>>>>>
>>>>>>> I do follow the newsletter, but am unlikely to go back and look into
>>>>>>> the past for changes/updates.
>>>>>>>
>>>>>>> Would suggest that things that get missed in one newsletter (a
>>>>>>> concrete example, Suneel's talks not mentioned in the newsletter) would get
>>>>>>> published in the next iteration, rather than editing the past 'published'
>>>>>>> newsletter. Put another way, save editing the past for corrections (typos,
>>>>>>> things being incorrect). Else, I imagine that I'm unlikely to catch a
>>>>>>> great announcement that warranted being in the newsletter in the first
>>>>>>> place. This certainly works better with a regular/frequent release
>>>>>>> cadence, like we arrived at for version releases (then, if something misses
>>>>>>> one cut, it is not too big a deal, as the next release is coming soon).
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Mar 6, 2019 at 12:50 PM Melissa Pashniak <
>>>>>>> melissapa@google.com> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> For step #2 (publishing onto the website), I think it would be good
>>>>>>>> to stay consistent with our existing workflows if possible. Rather than
>>>>>>>> using an external tool, what about:
>>>>>>>>
>>>>>>>> After a google doc newsletter draft is ready, convert it into a
>>>>>>>> standard markdown file and put it into our GitHub repo, perhaps in a new
>>>>>>>> newsletter directory in the website community directory [1]. These would be
>>>>>>>> listed for browsing on a Newsletters page as mentioned in step #4. People
>>>>>>>> can then just open a PR to add missing things to the pages later, and the
>>>>>>>> newsletter will be automatically updated on the website through our
>>>>>>>> standard website workflow. It also avoids the potential issue of the source
>>>>>>>> google docs disappearing in the future, as they are stored in a community
>>>>>>>> location.
>>>>>>>>
>>>>>>>> [1]
>>>>>>>> https://github.com/apache/beam/tree/master/website/src/community
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Mar 6, 2019 at 10:36 AM Rose Nguyen <rt...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I think that would be a great idea to change formats to help with
>>>>>>>>> distribution. I'm open to suggestions! I'm currently using a Google doc to
>>>>>>>>> collect and edit, then copy/paste sending the newsletter out directly, based
>>>>>>>>> on an interpretation of this discussion
>>>>>>>>> <https://lists.apache.org/thread.html/1f638eae43fe8abcb2f8752141c96d3dbdac86a583e0790044eea727@%3Cdev.beam.apache.org%3E>
>>>>>>>>> .
>>>>>>>>>
>>>>>>>>> How about this doc->website->Beam site workflow?:
>>>>>>>>>
>>>>>>>>> 1. The same usual newsletter [CALL FOR ITEMS] where you can
>>>>>>>>> contribute to the google doc, with soft deadlines for when I'll publish.
>>>>>>>>> 2. I'll publish the doc itself onto a website.
>>>>>>>>> 3. The newsletter is mailed out in the same way, but now with
>>>>>>>>> a shareable website link.
>>>>>>>>> 4. We'll keep an index of archived newsletter web pages on the
>>>>>>>>> Beam site, under the Community tab.
>>>>>>>>> 5. If you want to submit more content after the soft deadline,
>>>>>>>>> add it to the google doc and let me know to republish. I don't want to make
>>>>>>>>> the publication changes automatic because that leaves us open to tampering.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> This process is more laggy, so I'd suggest doing a 2 month vs
>>>>>>>>> monthly newsletter cadence. If we're happy with this idea, I'll send in a
>>>>>>>>> website PR for a new "Newsletter" left nav item under Community.
>>>>>>>>>
>>>>>>>>> Here's an example of a published newsletter: Apache Beam
>>>>>>>>> February-March 2019
>>>>>>>>> <https://gdoc.pub/doc/e/2PACX-1vTQIS4WkxV-HpgX5Lb6q05g4-wuIVcYd82123Mp4Y6q9fMv6Ynwd-l7dI4TrMyCrKilyU-YsoitbnZB>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> - This link is permanent unless the principal google doc is
>>>>>>>>> deleted.
>>>>>>>>> - Changes to the google doc after web publication are not
>>>>>>>>> automatically published on the website to protect the information integrity.
>>>>>>>>> - Republishing is quick and easy for me if you let me know
>>>>>>>>> you've added more.
>>>>>>>>> - I'll improve the formatting later if we go with this route.
>>>>>>>>>
>>>>>>>>> Any thoughts?
>>>>>>>>>
>>>>>>>>> On Wed, Mar 6, 2019 at 6:13 AM Thomas Weise <th...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Similar to blog posts. A link that can be shared would also help
>>>>>>>>>> to distribute over other channels, such as Twitter.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Mar 6, 2019, 6:06 AM Ismaël Mejía <ie...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> We should have these newsletters published somewhere with a
>>>>>>>>>>> fixed URL so we can add missing updates, I have at least missed putting
>>>>>>>>>>> stuff in the last 3 ones just to realize when the newsletter has been
>>>>>>>>>>> already 'published' (and sadly interesting features like zstd have not been
>>>>>>>>>>> announced because of this).
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Mar 6, 2019 at 2:48 PM Etienne Chauchot <
>>>>>>>>>>> echauchot@apache.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>> I would add in what's been done:
>>>>>>>>>>>>
>>>>>>>>>>>> Work on cassandraIO (Etienne Chauchot, Mathieu Blanchard,
>>>>>>>>>>>> Frank Shahar) : refactorings, bugfixes, new where clause, security fix
>>>>>>>>>>>>
>>>>>>>>>>>> Etienne
>>>>>>>>>>>>
>>>>>>>>>>>> Le lundi 04 mars 2019 à 18:36 +0100, Suneel Marthi a écrit :
>>>>>>>>>>>>
>>>>>>>>>>>> Is this the final draft? - we had 2 beam talks at Big Data Tech
>>>>>>>>>>>> Warsaw last Wednesday - I can send the updates offline.
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Mar 4, 2019 at 6:16 PM Rose Nguyen <rt...@google.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> [image: Beam.png]
>>>>>>>>>>>>
>>>>>>>>>>>> February-March 2019 | Newsletter
>>>>>>>>>>>>
>>>>>>>>>>>> What’s been done
>>>>>>>>>>>>
>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>
>>>>>>>>>>>> Apache Beam 2.10.0 released (by: many contributors)
>>>>>>>>>>>>
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Download the release here.
>>>>>>>>>>>> <https://beam.apache.org/get-started/downloads/>
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> See the blog post
>>>>>>>>>>>> <https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html>
>>>>>>>>>>>> for more details.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Apache Beam awarded the 2019 Technology of the Year Award!
>>>>>>>>>>>>
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> InfoWorld just awarded Beam the 2019 Technology of the Year
>>>>>>>>>>>> Award.
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> See this article
>>>>>>>>>>>> <https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html?nsdr=true>
>>>>>>>>>>>> for more details.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Kettle Beam 0.5 released with support for flink (by: Matt
>>>>>>>>>>>> Casters)
>>>>>>>>>>>>
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Kettle now supports Apache Flink as well as Cloud Dataflow
>>>>>>>>>>>> and Spark.
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> See Matt’s Blog
>>>>>>>>>>>> <http://sandbox.kettle.be/wordpress/index.php/2019/02/24/kettle-beam-update-0-5-0/>
>>>>>>>>>>>> for more details.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> What we’re working on...
>>>>>>>>>>>>
>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>
>>>>>>>>>>>> Apache Beam 2.11.0 release (by: many contributors)
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Hive Metastore Table provider for SQL (by: Anton Kedin)
>>>>>>>>>>>>
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Support for plugging table providers through Beam SQL API
>>>>>>>>>>>> to allow obtaining table schemas from external sources.
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> See the PR <https://github.com/apache/beam/pull/7746> for
>>>>>>>>>>>> more details.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> User Defined Coders for the Beam Go SDK (by: Robert Burke)
>>>>>>>>>>>>
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Working on expanding the variety of user defined types that
>>>>>>>>>>>> can be a member of a PCollection in the Go SDK.
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> See BEAM-3306
>>>>>>>>>>>> <https://issues.apache.org/jira/browse/BEAM-3306> for more
>>>>>>>>>>>> details.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen, Mark
>>>>>>>>>>>> Liu, Robbe Sneyders, Juta Staes, Valentyn Tymofieiev)
>>>>>>>>>>>>
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Beam 2.11.0 is the first release offering partial Python 3
>>>>>>>>>>>> support.
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Many thanks to all contributors who helped to reach this
>>>>>>>>>>>> milestone.
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> IO availablility on Python 3 is currently limited and only
>>>>>>>>>>>> Python 3.5 version has been tested extensively.
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Stay tuned on BEAM-1251 for more details.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Notebooks for quickstarts and custom I/O (by: David Cavazos)
>>>>>>>>>>>>
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Adding IPython notebooks and snippets
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> See [BEAM-6557] <https://github.com/apache/beam/pull/7679>
>>>>>>>>>>>> for more details.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> New members
>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>
>>>>>>>>>>>> New PMC member!
>>>>>>>>>>>>
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Etienne Chauchot, Nantes, France
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> New Committers!
>>>>>>>>>>>>
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Gleb Kanterov, Stockholm, Sweden
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Michael Luckey
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> New Contributors!
>>>>>>>>>>>>
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Kyle Weaver, San Francisco, CA
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Would like to help begin implementing portability
>>>>>>>>>>>> support for the Spark runner
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Tanay Tummapalli, Delhi, India
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Would like to contribute to Open Source this summer as
>>>>>>>>>>>> part of Google Summer of Code
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Brian Hulette, Seattle, WA
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Contributing to Beam Portability
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Michał Walenia, Warsaw, Poland
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Working on integration and load testing
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Daniel Chen, San Francisco, CA
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Working on Beam Samza runner
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Talks & meetups
>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Plugin Machine Intelligence and Apache Beam with Pentaho - Feb
>>>>>>>>>>>> 7 @ London
>>>>>>>>>>>>
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Watch the How to Run Kettle on Apache Beam video here
>>>>>>>>>>>> <https://skillsmatter.com/skillscasts/13405-how-to-run-kettle-on-apache-beam#video>.
>>>>>>>>>>>>
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> See event details here
>>>>>>>>>>>> <https://www.meetup.com/Pentaho-London-User-Group/events/256773962/>
>>>>>>>>>>>> ..
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @ San
>>>>>>>>>>>> Francisco, CA
>>>>>>>>>>>>
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Organized by Thomas Weise and Austin Bennet, with speakers
>>>>>>>>>>>> Tyler Akidau, Robert Crowe, Thomas Weise and Amar Pai
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> See event details here
>>>>>>>>>>>> <https://www.meetup.com/San-Francisco-Apache-Beam/events/257482350/>
>>>>>>>>>>>> and the slides for these presentation: Overview of Apache
>>>>>>>>>>>> Beam and TensorFlow Transform (TFX) with Apache Beam
>>>>>>>>>>>> <http://s.apache.org/beam-intro-feb-2019>, Python Streaming
>>>>>>>>>>>> Pipelines with Beam on Flink
>>>>>>>>>>>> <http://go.lyft.com/python-flink-beam-meetup-2019>, Dynamic
>>>>>>>>>>>> pricing of Lyft rides using streaming
>>>>>>>>>>>> <https://www.slideshare.net/AmarPai2/dynamic-pricing-of-lyft-rides-using-streaming>
>>>>>>>>>>>>
>>>>>>>>>>>> .
>>>>>>>>>>>>
>>>>>>>>>>>> Flink meetup - Feb 21@ Seattle, WA
>>>>>>>>>>>>
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Speakers from Alibaba, Google, and Uber gave talks about
>>>>>>>>>>>> Apache Flink with Hive, Tensorflow, Beam, and AthenaX.
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> See event details here
>>>>>>>>>>>> <https://www.meetup.com/seattle-flink/events/258723322/>
>>>>>>>>>>>> and presentations here
>>>>>>>>>>>> <https://www.slideshare.net/BowenLi9/presentations>.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Beam Summit Europe 2019 - June 19-20 @ Berlin
>>>>>>>>>>>>
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Beam Summit Europe 2019 will take place in Berlin on June
>>>>>>>>>>>> 19-20.
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Speaker CfP and other details to follow soon!
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> Twitter announcement!
>>>>>>>>>>>> <https://twitter.com/matthiasbaetens/status/1098854758893273088>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Resources
>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>
>>>>>>>>>>>> Apache Jira Beginner’s Guide (by: Daniel Oliveira)
>>>>>>>>>>>>
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> A guide
>>>>>>>>>>>> <https://cwiki.apache.org/confluence/display/BEAM/Beam+Jira+Beginner%27s+Guide>
>>>>>>>>>>>> to introduce Beam contributors to the basics of using the Apache Jira for
>>>>>>>>>>>> Beam development. Feedback welcomed!
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> An approach to community building from Apache Beam (by: Kenn
>>>>>>>>>>>> Knowles)
>>>>>>>>>>>>
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> The Apache Software Foundation has published committer
>>>>>>>>>>>> guidelines to help Beam's community building work.
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> See the post <https://blogs.apache.org/comdev/date/20190222>
>>>>>>>>>>>> on the ASF blog.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Exploring Beam SQL on Google Cloud Platform (by: Graham Polley)
>>>>>>>>>>>>
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> “In this article, I’ll dive into this new feature of Beam,
>>>>>>>>>>>> and see how it works by using a pipeline to read a data file from GCS,
>>>>>>>>>>>> transform it, and then perform a basic calculation on the values contained
>>>>>>>>>>>> in the file”.
>>>>>>>>>>>> -
>>>>>>>>>>>>
>>>>>>>>>>>> See article
>>>>>>>>>>>> <https://medium.com/weareservian/exploring-beam-sql-on-google-cloud-platform-b6c77f9b4af4>
>>>>>>>>>>>> and full source code
>>>>>>>>>>>> <https://github.com/polleyg/gcp-batch-ingestion-bigquery/blob/beam_sql/src/main/java/org/polleyg/BeamSQLMagic.java>
>>>>>>>>>>>> .
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> *Until Next Time!*
>>>>>>>>>>>> --
>>>>>>>>>>>> Rose Thị Nguyễn
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Rose Thị Nguyễn
>>>>>>>>>
>>>>>>>>
>>>>
>>>> --
>>>> Rose Thị Nguyễn
>>>>
>>>
>>>
>>> --
>>>
>>> *Aizhamal Nurmamat kyzy*
>>>
>>> Open Source Program Manager
>>>
>>> 646-355-9740 <(646)%20355-9740> Mobile
>>>
>>> 601 North 34th Street, Seattle, WA 98103
>>> <https://maps.google.com/?q=601+North+34th+Street,+Seattle,+WA+98103&entry=gmail&source=g>
>>>
>>>
>>>
Re: Apache Beam Newsletter - February/March 2019
Posted by Thomas Weise <th...@apache.org>.
+1 for single blog/news section
Also I wouldn't mind quarterly cadence, to provide more focus for folks to
contribute.
On Mon, Mar 11, 2019 at 6:35 PM Kenneth Knowles <ke...@apache.org> wrote:
> Could the newsletter be a blog entry? If you check out
> https://blogs.apache.org/ many of the posts are "Apache News Round-up".
> We could rename the "Blog" section to "News" if you ask me.
>
> Kenn
>
> On Mon, Mar 11, 2019 at 5:13 PM Aizhamal Nurmamat kyzy <
> aizhamal@google.com> wrote:
>
>> Hello everyone,
>>
>> I had a chat with Rose on how I can support the effort and keep sending
>> the newsletters on a monthly basis. The new workflow would look as follows:
>>
>> 1. Send out the same [CALL FOR ITEMS] where you can contribute to the
>> Google doc with deadlines
>> 2. Edit the doc after the deadline
>> 3. Convert the file into Markdown
>> 4. Send in PR to add the file to Beam repo in Newsletter directory
>> 5. Have people send their fixes/updates through PRs
>>
>> In this effort, I can support Rose in steps 3 & 4.
>>
>> We would also need:
>>
>> - Create a Newsletter section under the Community tab
>> - Write guidelines on newsletter contributions
>> - Make a note about timing e.g. if upcoming event, then add to the
>> next newsletter
>>
>> How does that sound to you all?
>>
>>
>> On Wed, Mar 6, 2019 at 9:19 PM Rose Nguyen <rt...@google.com> wrote:
>>
>>> Good points. With the suggested workflow I think I can support a
>>> quarterly newsletter. I'm also happy to get more involvement from others to
>>> do this work and we can see what cadence that allows.
>>>
>>> On Wed, Mar 6, 2019 at 8:22 PM Kenneth Knowles <ke...@apache.org> wrote:
>>>
>>>> Good points Melissa & Austin.
>>>>
>>>> - archive in the repo & on the website
>>>> - put missed items on the next newsletter, so anyone following sees
>>>> them
>>>>
>>>> Kenn
>>>>
>>>> On Wed, Mar 6, 2019 at 3:26 PM Suneel Marthi <sm...@apache.org>
>>>> wrote:
>>>>
>>>>> I believe there was also a Beam workshop or working session in Warsaw
>>>>> last week.
>>>>>
>>>>> On Wed, Mar 6, 2019 at 6:20 PM Austin Bennett <
>>>>> whatwouldaustindo@gmail.com> wrote:
>>>>>
>>>>>> +1 for archive in our repo.
>>>>>>
>>>>>> I do follow the newsletter, but am unlikely to go back and look into
>>>>>> the past for changes/updates.
>>>>>>
>>>>>> Would suggest that things that get missed in one newsletter (a
>>>>>> concrete example, Suneel's talks not mentioned in the newsletter) would get
>>>>>> published in the next iteration, rather than editing the past 'published'
>>>>>> newsletter. Put another way, save editing the past for corrections (typos,
>>>>>> things being incorrect). Else, I imagine that I'm unlikely to catch a
>>>>>> great announcement that warranted being in the newsletter in the first
>>>>>> place. This certainly works better with a regular/frequent release
>>>>>> cadence, like we arrived at for version releases (then, if something misses
>>>>>> one cut, it is not too big a deal, as the next release is coming soon).
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Mar 6, 2019 at 12:50 PM Melissa Pashniak <
>>>>>> melissapa@google.com> wrote:
>>>>>>
>>>>>>>
>>>>>>> For step #2 (publishing onto the website), I think it would be good
>>>>>>> to stay consistent with our existing workflows if possible. Rather than
>>>>>>> using an external tool, what about:
>>>>>>>
>>>>>>> After a google doc newsletter draft is ready, convert it into a
>>>>>>> standard markdown file and put it into our GitHub repo, perhaps in a new
>>>>>>> newsletter directory in the website community directory [1]. These would be
>>>>>>> listed for browsing on a Newsletters page as mentioned in step #4. People
>>>>>>> can then just open a PR to add missing things to the pages later, and the
>>>>>>> newsletter will be automatically updated on the website through our
>>>>>>> standard website workflow. It also avoids the potential issue of the source
>>>>>>> google docs disappearing in the future, as they are stored in a community
>>>>>>> location.
>>>>>>>
>>>>>>> [1] https://github.com/apache/beam/tree/master/website/src/community
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Mar 6, 2019 at 10:36 AM Rose Nguyen <rt...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I think that would be a great idea to change formats to help with
>>>>>>>> distribution. I'm open to suggestions! I'm currently using a Google doc to
>>>>>>>> collect and edit, then copy/paste sending the newsletter out directly, based
>>>>>>>> on an interpretation of this discussion
>>>>>>>> <https://lists.apache.org/thread.html/1f638eae43fe8abcb2f8752141c96d3dbdac86a583e0790044eea727@%3Cdev.beam.apache.org%3E>
>>>>>>>> .
>>>>>>>>
>>>>>>>> How about this doc->website->Beam site workflow?:
>>>>>>>>
>>>>>>>> 1. The same usual newsletter [CALL FOR ITEMS] where you can
>>>>>>>> contribute to the google doc, with soft deadlines for when I'll publish.
>>>>>>>> 2. I'll publish the doc itself onto a website.
>>>>>>>> 3. The newsletter is mailed out in the same way, but now with a
>>>>>>>> shareable website link.
>>>>>>>> 4. We'll keep an index of archived newsletter web pages on the
>>>>>>>> Beam site, under the Community tab.
>>>>>>>> 5. If you want to submit more content after the soft deadline,
>>>>>>>> add it to the google doc and let me know to republish. I don't want to make
>>>>>>>> the publication changes automatic because that leaves us open to tampering.
>>>>>>>>
>>>>>>>>
>>>>>>>> This process is more laggy, so I'd suggest doing a 2 month vs
>>>>>>>> monthly newsletter cadence. If we're happy with this idea, I'll send in a
>>>>>>>> website PR for a new "Newsletter" left nav item under Community.
>>>>>>>>
>>>>>>>> Here's an example of a published newsletter: Apache Beam
>>>>>>>> February-March 2019
>>>>>>>> <https://gdoc.pub/doc/e/2PACX-1vTQIS4WkxV-HpgX5Lb6q05g4-wuIVcYd82123Mp4Y6q9fMv6Ynwd-l7dI4TrMyCrKilyU-YsoitbnZB>
>>>>>>>>
>>>>>>>>
>>>>>>>> - This link is permanent unless the principal google doc is
>>>>>>>> deleted.
>>>>>>>> - Changes to the google doc after web publication are not
>>>>>>>> automatically published on the website to protect the information integrity.
>>>>>>>> - Republishing is quick and easy for me if you let me know
>>>>>>>> you've added more.
>>>>>>>> - I'll improve the formatting later if we go with this route.
>>>>>>>>
>>>>>>>> Any thoughts?
>>>>>>>>
>>>>>>>> On Wed, Mar 6, 2019 at 6:13 AM Thomas Weise <th...@apache.org> wrote:
>>>>>>>>
>>>>>>>>> Similar to blog posts. A link that can be shared would also help
>>>>>>>>> to distribute over other channels, such as Twitter.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Mar 6, 2019, 6:06 AM Ismaël Mejía <ie...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> We should have these newsletters published somewhere with a fixed
>>>>>>>>>> URL so we can add missing updates, I have at least missed putting stuff in
>>>>>>>>>> the last 3 ones just to realize when the newsletter has been already
>>>>>>>>>> 'published' (and sadly interesting features like zstd have not been
>>>>>>>>>> announced because of this).
>>>>>>>>>>
>>>>>>>>>> On Wed, Mar 6, 2019 at 2:48 PM Etienne Chauchot <
>>>>>>>>>> echauchot@apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>> I would add in what's been done:
>>>>>>>>>>>
>>>>>>>>>>> Work on cassandraIO (Etienne Chauchot, Mathieu Blanchard,
>>>>>>>>>>> Frank Shahar) : refactorings, bugfixes, new where clause, security fix
>>>>>>>>>>>
>>>>>>>>>>> Etienne
>>>>>>>>>>>
>>>>>>>>>>> Le lundi 04 mars 2019 à 18:36 +0100, Suneel Marthi a écrit :
>>>>>>>>>>>
>>>>>>>>>>> Is this the final draft? - we had 2 beam talks at Big Data Tech
>>>>>>>>>>> Warsaw last Wednesday - I can send the updates offline.
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Mar 4, 2019 at 6:16 PM Rose Nguyen <rt...@google.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> [image: Beam.png]
>>>>>>>>>>>
>>>>>>>>>>> February-March 2019 | Newsletter
>>>>>>>>>>>
>>>>>>>>>>> What’s been done
>>>>>>>>>>>
>>>>>>>>>>> ------------------------------
>>>>>>>>>>>
>>>>>>>>>>> Apache Beam 2.10.0 released (by: many contributors)
>>>>>>>>>>>
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Download the release here.
>>>>>>>>>>> <https://beam.apache.org/get-started/downloads/>
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> See the blog post
>>>>>>>>>>> <https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html>
>>>>>>>>>>> for more details.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Apache Beam awarded the 2019 Technology of the Year Award!
>>>>>>>>>>>
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> InfoWorld just awarded Beam the 2019 Technology of the Year
>>>>>>>>>>> Award.
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> See this article
>>>>>>>>>>> <https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html?nsdr=true>
>>>>>>>>>>> for more details.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Kettle Beam 0.5 released with support for flink (by: Matt
>>>>>>>>>>> Casters)
>>>>>>>>>>>
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Kettle now supports Apache Flink as well as Cloud Dataflow
>>>>>>>>>>> and Spark.
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> See Matt’s Blog
>>>>>>>>>>> <http://sandbox.kettle.be/wordpress/index.php/2019/02/24/kettle-beam-update-0-5-0/>
>>>>>>>>>>> for more details.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> What we’re working on...
>>>>>>>>>>>
>>>>>>>>>>> ------------------------------
>>>>>>>>>>>
>>>>>>>>>>> Apache Beam 2.11.0 release (by: many contributors)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Hive Metastore Table provider for SQL (by: Anton Kedin)
>>>>>>>>>>>
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Support for plugging table providers through Beam SQL API to
>>>>>>>>>>> allow obtaining table schemas from external sources.
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> See the PR <https://github.com/apache/beam/pull/7746> for
>>>>>>>>>>> more details.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> User Defined Coders for the Beam Go SDK (by: Robert Burke)
>>>>>>>>>>>
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Working on expanding the variety of user defined types that
>>>>>>>>>>> can be a member of a PCollection in the Go SDK.
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> See BEAM-3306
>>>>>>>>>>> <https://issues.apache.org/jira/browse/BEAM-3306> for more
>>>>>>>>>>> details.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen, Mark
>>>>>>>>>>> Liu, Robbe Sneyders, Juta Staes, Valentyn Tymofieiev)
>>>>>>>>>>>
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Beam 2.11.0 is the first release offering partial Python 3
>>>>>>>>>>> support.
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Many thanks to all contributors who helped to reach this
>>>>>>>>>>> milestone.
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> IO availablility on Python 3 is currently limited and only
>>>>>>>>>>> Python 3.5 version has been tested extensively.
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Stay tuned on BEAM-1251 for more details.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Notebooks for quickstarts and custom I/O (by: David Cavazos)
>>>>>>>>>>>
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Adding IPython notebooks and snippets
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> See [BEAM-6557] <https://github.com/apache/beam/pull/7679>
>>>>>>>>>>> for more details.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> New members
>>>>>>>>>>> ------------------------------
>>>>>>>>>>>
>>>>>>>>>>> New PMC member!
>>>>>>>>>>>
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Etienne Chauchot, Nantes, France
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> New Committers!
>>>>>>>>>>>
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Gleb Kanterov, Stockholm, Sweden
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Michael Luckey
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> New Contributors!
>>>>>>>>>>>
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Kyle Weaver, San Francisco, CA
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Would like to help begin implementing portability support
>>>>>>>>>>> for the Spark runner
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Tanay Tummapalli, Delhi, India
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Would like to contribute to Open Source this summer as
>>>>>>>>>>> part of Google Summer of Code
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Brian Hulette, Seattle, WA
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Contributing to Beam Portability
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Michał Walenia, Warsaw, Poland
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Working on integration and load testing
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Daniel Chen, San Francisco, CA
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Working on Beam Samza runner
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Talks & meetups
>>>>>>>>>>> ------------------------------
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Plugin Machine Intelligence and Apache Beam with Pentaho - Feb 7
>>>>>>>>>>> @ London
>>>>>>>>>>>
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Watch the How to Run Kettle on Apache Beam video here
>>>>>>>>>>> <https://skillsmatter.com/skillscasts/13405-how-to-run-kettle-on-apache-beam#video>.
>>>>>>>>>>>
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> See event details here
>>>>>>>>>>> <https://www.meetup.com/Pentaho-London-User-Group/events/256773962/>
>>>>>>>>>>> ..
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @ San
>>>>>>>>>>> Francisco, CA
>>>>>>>>>>>
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Organized by Thomas Weise and Austin Bennet, with speakers
>>>>>>>>>>> Tyler Akidau, Robert Crowe, Thomas Weise and Amar Pai
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> See event details here
>>>>>>>>>>> <https://www.meetup.com/San-Francisco-Apache-Beam/events/257482350/>
>>>>>>>>>>> and the slides for these presentation: Overview of Apache
>>>>>>>>>>> Beam and TensorFlow Transform (TFX) with Apache Beam
>>>>>>>>>>> <http://s.apache.org/beam-intro-feb-2019>, Python Streaming
>>>>>>>>>>> Pipelines with Beam on Flink
>>>>>>>>>>> <http://go.lyft.com/python-flink-beam-meetup-2019>, Dynamic
>>>>>>>>>>> pricing of Lyft rides using streaming
>>>>>>>>>>> <https://www.slideshare.net/AmarPai2/dynamic-pricing-of-lyft-rides-using-streaming>
>>>>>>>>>>>
>>>>>>>>>>> .
>>>>>>>>>>>
>>>>>>>>>>> Flink meetup - Feb 21@ Seattle, WA
>>>>>>>>>>>
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Speakers from Alibaba, Google, and Uber gave talks about
>>>>>>>>>>> Apache Flink with Hive, Tensorflow, Beam, and AthenaX.
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> See event details here
>>>>>>>>>>> <https://www.meetup.com/seattle-flink/events/258723322/> and
>>>>>>>>>>> presentations here
>>>>>>>>>>> <https://www.slideshare.net/BowenLi9/presentations>.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Beam Summit Europe 2019 - June 19-20 @ Berlin
>>>>>>>>>>>
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Beam Summit Europe 2019 will take place in Berlin on June
>>>>>>>>>>> 19-20.
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Speaker CfP and other details to follow soon!
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> Twitter announcement!
>>>>>>>>>>> <https://twitter.com/matthiasbaetens/status/1098854758893273088>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Resources
>>>>>>>>>>> ------------------------------
>>>>>>>>>>>
>>>>>>>>>>> Apache Jira Beginner’s Guide (by: Daniel Oliveira)
>>>>>>>>>>>
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> A guide
>>>>>>>>>>> <https://cwiki.apache.org/confluence/display/BEAM/Beam+Jira+Beginner%27s+Guide>
>>>>>>>>>>> to introduce Beam contributors to the basics of using the Apache Jira for
>>>>>>>>>>> Beam development. Feedback welcomed!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> An approach to community building from Apache Beam (by: Kenn
>>>>>>>>>>> Knowles)
>>>>>>>>>>>
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> The Apache Software Foundation has published committer
>>>>>>>>>>> guidelines to help Beam's community building work.
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> See the post <https://blogs.apache.org/comdev/date/20190222>
>>>>>>>>>>> on the ASF blog.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Exploring Beam SQL on Google Cloud Platform (by: Graham Polley)
>>>>>>>>>>>
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> “In this article, I’ll dive into this new feature of Beam,
>>>>>>>>>>> and see how it works by using a pipeline to read a data file from GCS,
>>>>>>>>>>> transform it, and then perform a basic calculation on the values contained
>>>>>>>>>>> in the file”.
>>>>>>>>>>> -
>>>>>>>>>>>
>>>>>>>>>>> See article
>>>>>>>>>>> <https://medium.com/weareservian/exploring-beam-sql-on-google-cloud-platform-b6c77f9b4af4>
>>>>>>>>>>> and full source code
>>>>>>>>>>> <https://github.com/polleyg/gcp-batch-ingestion-bigquery/blob/beam_sql/src/main/java/org/polleyg/BeamSQLMagic.java>
>>>>>>>>>>> .
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> *Until Next Time!*
>>>>>>>>>>> --
>>>>>>>>>>> Rose Thị Nguyễn
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Rose Thị Nguyễn
>>>>>>>>
>>>>>>>
>>>
>>> --
>>> Rose Thị Nguyễn
>>>
>>
>>
>> --
>>
>> *Aizhamal Nurmamat kyzy*
>>
>> Open Source Program Manager
>>
>> 646-355-9740 Mobile
>>
>> 601 North 34th Street, Seattle, WA 98103
>>
>>
>>
Re: Apache Beam Newsletter - February/March 2019
Posted by Kenneth Knowles <ke...@apache.org>.
Could the newsletter be a blog entry? If you check out
https://blogs.apache.org/ many of the posts are "Apache News Round-up". We
could rename the "Blog" section to "News" if you ask me.
Kenn
On Mon, Mar 11, 2019 at 5:13 PM Aizhamal Nurmamat kyzy <ai...@google.com>
wrote:
> Hello everyone,
>
> I had a chat with Rose on how I can support the effort and keep sending
> the newsletters on a monthly basis. The new workflow would look as follows:
>
> 1. Send out the same [CALL FOR ITEMS] where you can contribute to the
> Google doc with deadlines
> 2. Edit the doc after the deadline
> 3. Convert the file into Markdown
> 4. Send in PR to add the file to Beam repo in Newsletter directory
> 5. Have people send their fixes/updates through PRs
>
> In this effort, I can support Rose in steps 3 & 4.
>
> We would also need:
>
> - Create a Newsletter section under the Community tab
> - Write guidelines on newsletter contributions
> - Make a note about timing e.g. if upcoming event, then add to the
> next newsletter
>
> How does that sound to you all?
>
>
> On Wed, Mar 6, 2019 at 9:19 PM Rose Nguyen <rt...@google.com> wrote:
>
>> Good points. With the suggested workflow I think I can support a
>> quarterly newsletter. I'm also happy to get more involvement from others to
>> do this work and we can see what cadence that allows.
>>
>> On Wed, Mar 6, 2019 at 8:22 PM Kenneth Knowles <ke...@apache.org> wrote:
>>
>>> Good points Melissa & Austin.
>>>
>>> - archive in the repo & on the website
>>> - put missed items on the next newsletter, so anyone following sees them
>>>
>>> Kenn
>>>
>>> On Wed, Mar 6, 2019 at 3:26 PM Suneel Marthi <sm...@apache.org> wrote:
>>>
>>>> I believe there was also a Beam workshop or working session in Warsaw
>>>> last week.
>>>>
>>>> On Wed, Mar 6, 2019 at 6:20 PM Austin Bennett <
>>>> whatwouldaustindo@gmail.com> wrote:
>>>>
>>>>> +1 for archive in our repo.
>>>>>
>>>>> I do follow the newsletter, but am unlikely to go back and look into
>>>>> the past for changes/updates.
>>>>>
>>>>> Would suggest that things that get missed in one newsletter (a
>>>>> concrete example, Suneel's talks not mentioned in the newsletter) would get
>>>>> published in the next iteration, rather than editing the past 'published'
>>>>> newsletter. Put another way, save editing the past for corrections (typos,
>>>>> things being incorrect). Else, I imagine that I'm unlikely to catch a
>>>>> great announcement that warranted being in the newsletter in the first
>>>>> place. This certainly works better with a regular/frequent release
>>>>> cadence, like we arrived at for version releases (then, if something misses
>>>>> one cut, it is not too big a deal, as the next release is coming soon).
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Mar 6, 2019 at 12:50 PM Melissa Pashniak <me...@google.com>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>> For step #2 (publishing onto the website), I think it would be good
>>>>>> to stay consistent with our existing workflows if possible. Rather than
>>>>>> using an external tool, what about:
>>>>>>
>>>>>> After a google doc newsletter draft is ready, convert it into a
>>>>>> standard markdown file and put it into our GitHub repo, perhaps in a new
>>>>>> newsletter directory in the website community directory [1]. These would be
>>>>>> listed for browsing on a Newsletters page as mentioned in step #4. People
>>>>>> can then just open a PR to add missing things to the pages later, and the
>>>>>> newsletter will be automatically updated on the website through our
>>>>>> standard website workflow. It also avoids the potential issue of the source
>>>>>> google docs disappearing in the future, as they are stored in a community
>>>>>> location.
>>>>>>
>>>>>> [1] https://github.com/apache/beam/tree/master/website/src/community
>>>>>>
>>>>>>
>>>>>> On Wed, Mar 6, 2019 at 10:36 AM Rose Nguyen <rt...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I think that would be a great idea to change formats to help with
>>>>>>> distribution. I'm open to suggestions! I'm currently using a Google doc to
>>>>>>> collect and edit, then copy/paste sending the newsletter out directly, based
>>>>>>> on an interpretation of this discussion
>>>>>>> <https://lists.apache.org/thread.html/1f638eae43fe8abcb2f8752141c96d3dbdac86a583e0790044eea727@%3Cdev.beam.apache.org%3E>
>>>>>>> .
>>>>>>>
>>>>>>> How about this doc->website->Beam site workflow?:
>>>>>>>
>>>>>>> 1. The same usual newsletter [CALL FOR ITEMS] where you can
>>>>>>> contribute to the google doc, with soft deadlines for when I'll publish.
>>>>>>> 2. I'll publish the doc itself onto a website.
>>>>>>> 3. The newsletter is mailed out in the same way, but now with a
>>>>>>> shareable website link.
>>>>>>> 4. We'll keep an index of archived newsletter web pages on the
>>>>>>> Beam site, under the Community tab.
>>>>>>> 5. If you want to submit more content after the soft deadline,
>>>>>>> add it to the google doc and let me know to republish. I don't want to make
>>>>>>> the publication changes automatic because that leaves us open to tampering.
>>>>>>>
>>>>>>>
>>>>>>> This process is more laggy, so I'd suggest doing a 2 month vs
>>>>>>> monthly newsletter cadence. If we're happy with this idea, I'll send in a
>>>>>>> website PR for a new "Newsletter" left nav item under Community.
>>>>>>>
>>>>>>> Here's an example of a published newsletter: Apache Beam
>>>>>>> February-March 2019
>>>>>>> <https://gdoc.pub/doc/e/2PACX-1vTQIS4WkxV-HpgX5Lb6q05g4-wuIVcYd82123Mp4Y6q9fMv6Ynwd-l7dI4TrMyCrKilyU-YsoitbnZB>
>>>>>>>
>>>>>>>
>>>>>>> - This link is permanent unless the principal google doc is
>>>>>>> deleted.
>>>>>>> - Changes to the google doc after web publication are not
>>>>>>> automatically published on the website to protect the information integrity.
>>>>>>> - Republishing is quick and easy for me if you let me know
>>>>>>> you've added more.
>>>>>>> - I'll improve the formatting later if we go with this route.
>>>>>>>
>>>>>>> Any thoughts?
>>>>>>>
>>>>>>> On Wed, Mar 6, 2019 at 6:13 AM Thomas Weise <th...@apache.org> wrote:
>>>>>>>
>>>>>>>> Similar to blog posts. A link that can be shared would also help to
>>>>>>>> distribute over other channels, such as Twitter.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Mar 6, 2019, 6:06 AM Ismaël Mejía <ie...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> We should have these newsletters published somewhere with a fixed
>>>>>>>>> URL so we can add missing updates, I have at least missed putting stuff in
>>>>>>>>> the last 3 ones just to realize when the newsletter has been already
>>>>>>>>> 'published' (and sadly interesting features like zstd have not been
>>>>>>>>> announced because of this).
>>>>>>>>>
>>>>>>>>> On Wed, Mar 6, 2019 at 2:48 PM Etienne Chauchot <
>>>>>>>>> echauchot@apache.org> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>> I would add in what's been done:
>>>>>>>>>>
>>>>>>>>>> Work on cassandraIO (Etienne Chauchot, Mathieu Blanchard,
>>>>>>>>>> Frank Shahar) : refactorings, bugfixes, new where clause, security fix
>>>>>>>>>>
>>>>>>>>>> Etienne
>>>>>>>>>>
>>>>>>>>>> Le lundi 04 mars 2019 à 18:36 +0100, Suneel Marthi a écrit :
>>>>>>>>>>
>>>>>>>>>> Is this the final draft? - we had 2 beam talks at Big Data Tech
>>>>>>>>>> Warsaw last Wednesday - I can send the updates offline.
>>>>>>>>>>
>>>>>>>>>> On Mon, Mar 4, 2019 at 6:16 PM Rose Nguyen <rt...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> [image: Beam.png]
>>>>>>>>>>
>>>>>>>>>> February-March 2019 | Newsletter
>>>>>>>>>>
>>>>>>>>>> What’s been done
>>>>>>>>>>
>>>>>>>>>> ------------------------------
>>>>>>>>>>
>>>>>>>>>> Apache Beam 2.10.0 released (by: many contributors)
>>>>>>>>>>
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Download the release here.
>>>>>>>>>> <https://beam.apache.org/get-started/downloads/>
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> See the blog post
>>>>>>>>>> <https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html>
>>>>>>>>>> for more details.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Apache Beam awarded the 2019 Technology of the Year Award!
>>>>>>>>>>
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> InfoWorld just awarded Beam the 2019 Technology of the Year
>>>>>>>>>> Award.
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> See this article
>>>>>>>>>> <https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html?nsdr=true>
>>>>>>>>>> for more details.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Kettle Beam 0.5 released with support for flink (by: Matt Casters)
>>>>>>>>>>
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Kettle now supports Apache Flink as well as Cloud Dataflow
>>>>>>>>>> and Spark.
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> See Matt’s Blog
>>>>>>>>>> <http://sandbox.kettle.be/wordpress/index.php/2019/02/24/kettle-beam-update-0-5-0/>
>>>>>>>>>> for more details.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> What we’re working on...
>>>>>>>>>>
>>>>>>>>>> ------------------------------
>>>>>>>>>>
>>>>>>>>>> Apache Beam 2.11.0 release (by: many contributors)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Hive Metastore Table provider for SQL (by: Anton Kedin)
>>>>>>>>>>
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Support for plugging table providers through Beam SQL API to
>>>>>>>>>> allow obtaining table schemas from external sources.
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> See the PR <https://github.com/apache/beam/pull/7746> for
>>>>>>>>>> more details.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> User Defined Coders for the Beam Go SDK (by: Robert Burke)
>>>>>>>>>>
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Working on expanding the variety of user defined types that
>>>>>>>>>> can be a member of a PCollection in the Go SDK.
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> See BEAM-3306
>>>>>>>>>> <https://issues.apache.org/jira/browse/BEAM-3306> for more
>>>>>>>>>> details.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen, Mark
>>>>>>>>>> Liu, Robbe Sneyders, Juta Staes, Valentyn Tymofieiev)
>>>>>>>>>>
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Beam 2.11.0 is the first release offering partial Python 3
>>>>>>>>>> support.
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Many thanks to all contributors who helped to reach this
>>>>>>>>>> milestone.
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> IO availablility on Python 3 is currently limited and only
>>>>>>>>>> Python 3.5 version has been tested extensively.
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Stay tuned on BEAM-1251 for more details.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Notebooks for quickstarts and custom I/O (by: David Cavazos)
>>>>>>>>>>
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Adding IPython notebooks and snippets
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> See [BEAM-6557] <https://github.com/apache/beam/pull/7679>
>>>>>>>>>> for more details.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> New members
>>>>>>>>>> ------------------------------
>>>>>>>>>>
>>>>>>>>>> New PMC member!
>>>>>>>>>>
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Etienne Chauchot, Nantes, France
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> New Committers!
>>>>>>>>>>
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Gleb Kanterov, Stockholm, Sweden
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Michael Luckey
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> New Contributors!
>>>>>>>>>>
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Kyle Weaver, San Francisco, CA
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Would like to help begin implementing portability support
>>>>>>>>>> for the Spark runner
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Tanay Tummapalli, Delhi, India
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Would like to contribute to Open Source this summer as
>>>>>>>>>> part of Google Summer of Code
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Brian Hulette, Seattle, WA
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Contributing to Beam Portability
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Michał Walenia, Warsaw, Poland
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Working on integration and load testing
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Daniel Chen, San Francisco, CA
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Working on Beam Samza runner
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Talks & meetups
>>>>>>>>>> ------------------------------
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Plugin Machine Intelligence and Apache Beam with Pentaho - Feb 7
>>>>>>>>>> @ London
>>>>>>>>>>
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Watch the How to Run Kettle on Apache Beam video here
>>>>>>>>>> <https://skillsmatter.com/skillscasts/13405-how-to-run-kettle-on-apache-beam#video>.
>>>>>>>>>>
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> See event details here
>>>>>>>>>> <https://www.meetup.com/Pentaho-London-User-Group/events/256773962/>
>>>>>>>>>> ..
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @ San
>>>>>>>>>> Francisco, CA
>>>>>>>>>>
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Organized by Thomas Weise and Austin Bennet, with speakers
>>>>>>>>>> Tyler Akidau, Robert Crowe, Thomas Weise and Amar Pai
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> See event details here
>>>>>>>>>> <https://www.meetup.com/San-Francisco-Apache-Beam/events/257482350/>
>>>>>>>>>> and the slides for these presentation: Overview of Apache
>>>>>>>>>> Beam and TensorFlow Transform (TFX) with Apache Beam
>>>>>>>>>> <http://s.apache.org/beam-intro-feb-2019>, Python Streaming
>>>>>>>>>> Pipelines with Beam on Flink
>>>>>>>>>> <http://go.lyft.com/python-flink-beam-meetup-2019>, Dynamic
>>>>>>>>>> pricing of Lyft rides using streaming
>>>>>>>>>> <https://www.slideshare.net/AmarPai2/dynamic-pricing-of-lyft-rides-using-streaming>
>>>>>>>>>>
>>>>>>>>>> .
>>>>>>>>>>
>>>>>>>>>> Flink meetup - Feb 21@ Seattle, WA
>>>>>>>>>>
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Speakers from Alibaba, Google, and Uber gave talks about
>>>>>>>>>> Apache Flink with Hive, Tensorflow, Beam, and AthenaX.
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> See event details here
>>>>>>>>>> <https://www.meetup.com/seattle-flink/events/258723322/> and
>>>>>>>>>> presentations here
>>>>>>>>>> <https://www.slideshare.net/BowenLi9/presentations>.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Beam Summit Europe 2019 - June 19-20 @ Berlin
>>>>>>>>>>
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Beam Summit Europe 2019 will take place in Berlin on June
>>>>>>>>>> 19-20.
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Speaker CfP and other details to follow soon!
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> Twitter announcement!
>>>>>>>>>> <https://twitter.com/matthiasbaetens/status/1098854758893273088>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Resources
>>>>>>>>>> ------------------------------
>>>>>>>>>>
>>>>>>>>>> Apache Jira Beginner’s Guide (by: Daniel Oliveira)
>>>>>>>>>>
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> A guide
>>>>>>>>>> <https://cwiki.apache.org/confluence/display/BEAM/Beam+Jira+Beginner%27s+Guide>
>>>>>>>>>> to introduce Beam contributors to the basics of using the Apache Jira for
>>>>>>>>>> Beam development. Feedback welcomed!
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> An approach to community building from Apache Beam (by: Kenn
>>>>>>>>>> Knowles)
>>>>>>>>>>
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> The Apache Software Foundation has published committer
>>>>>>>>>> guidelines to help Beam's community building work.
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> See the post <https://blogs.apache.org/comdev/date/20190222>
>>>>>>>>>> on the ASF blog.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Exploring Beam SQL on Google Cloud Platform (by: Graham Polley)
>>>>>>>>>>
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> “In this article, I’ll dive into this new feature of Beam,
>>>>>>>>>> and see how it works by using a pipeline to read a data file from GCS,
>>>>>>>>>> transform it, and then perform a basic calculation on the values contained
>>>>>>>>>> in the file”.
>>>>>>>>>> -
>>>>>>>>>>
>>>>>>>>>> See article
>>>>>>>>>> <https://medium.com/weareservian/exploring-beam-sql-on-google-cloud-platform-b6c77f9b4af4>
>>>>>>>>>> and full source code
>>>>>>>>>> <https://github.com/polleyg/gcp-batch-ingestion-bigquery/blob/beam_sql/src/main/java/org/polleyg/BeamSQLMagic.java>
>>>>>>>>>> .
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> *Until Next Time!*
>>>>>>>>>> --
>>>>>>>>>> Rose Thị Nguyễn
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Rose Thị Nguyễn
>>>>>>>
>>>>>>
>>
>> --
>> Rose Thị Nguyễn
>>
>
>
> --
>
> *Aizhamal Nurmamat kyzy*
>
> Open Source Program Manager
>
> 646-355-9740 Mobile
>
> 601 North 34th Street, Seattle, WA 98103
>
>
>
Re: Apache Beam Newsletter - February/March 2019
Posted by Aizhamal Nurmamat kyzy <ai...@google.com>.
Hello everyone,
I had a chat with Rose on how I can support the effort and keep sending the
newsletters on a monthly basis. The new workflow would look as follows:
1. Send out the same [CALL FOR ITEMS] where you can contribute to the
Google doc with deadlines
2. Edit the doc after the deadline
3. Convert the file into Markdown
4. Send in PR to add the file to Beam repo in Newsletter directory
5. Have people send their fixes/updates through PRs
In this effort, I can support Rose in steps 3 & 4.
We would also need:
- Create a Newsletter section under the Community tab
- Write guidelines on newsletter contributions
- Make a note about timing e.g. if upcoming event, then add to the next
newsletter
How does that sound to you all?
On Wed, Mar 6, 2019 at 9:19 PM Rose Nguyen <rt...@google.com> wrote:
> Good points. With the suggested workflow I think I can support a quarterly
> newsletter. I'm also happy to get more involvement from others to do this
> work and we can see what cadence that allows.
>
> On Wed, Mar 6, 2019 at 8:22 PM Kenneth Knowles <ke...@apache.org> wrote:
>
>> Good points Melissa & Austin.
>>
>> - archive in the repo & on the website
>> - put missed items on the next newsletter, so anyone following sees them
>>
>> Kenn
>>
>> On Wed, Mar 6, 2019 at 3:26 PM Suneel Marthi <sm...@apache.org> wrote:
>>
>>> I believe there was also a Beam workshop or working session in Warsaw
>>> last week.
>>>
>>> On Wed, Mar 6, 2019 at 6:20 PM Austin Bennett <
>>> whatwouldaustindo@gmail.com> wrote:
>>>
>>>> +1 for archive in our repo.
>>>>
>>>> I do follow the newsletter, but am unlikely to go back and look into
>>>> the past for changes/updates.
>>>>
>>>> Would suggest that things that get missed in one newsletter (a concrete
>>>> example, Suneel's talks not mentioned in the newsletter) would get
>>>> published in the next iteration, rather than editing the past 'published'
>>>> newsletter. Put another way, save editing the past for corrections (typos,
>>>> things being incorrect). Else, I imagine that I'm unlikely to catch a
>>>> great announcement that warranted being in the newsletter in the first
>>>> place. This certainly works better with a regular/frequent release
>>>> cadence, like we arrived at for version releases (then, if something misses
>>>> one cut, it is not too big a deal, as the next release is coming soon).
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, Mar 6, 2019 at 12:50 PM Melissa Pashniak <me...@google.com>
>>>> wrote:
>>>>
>>>>>
>>>>> For step #2 (publishing onto the website), I think it would be good to
>>>>> stay consistent with our existing workflows if possible. Rather than using
>>>>> an external tool, what about:
>>>>>
>>>>> After a google doc newsletter draft is ready, convert it into a
>>>>> standard markdown file and put it into our GitHub repo, perhaps in a new
>>>>> newsletter directory in the website community directory [1]. These would be
>>>>> listed for browsing on a Newsletters page as mentioned in step #4. People
>>>>> can then just open a PR to add missing things to the pages later, and the
>>>>> newsletter will be automatically updated on the website through our
>>>>> standard website workflow. It also avoids the potential issue of the source
>>>>> google docs disappearing in the future, as they are stored in a community
>>>>> location.
>>>>>
>>>>> [1] https://github.com/apache/beam/tree/master/website/src/community
>>>>>
>>>>>
>>>>> On Wed, Mar 6, 2019 at 10:36 AM Rose Nguyen <rt...@google.com>
>>>>> wrote:
>>>>>
>>>>>> I think that would be a great idea to change formats to help with
>>>>>> distribution. I'm open to suggestions! I'm currently using a Google doc to
>>>>>> collect and edit, then copy/paste sending the newsletter out directly, based
>>>>>> on an interpretation of this discussion
>>>>>> <https://lists.apache.org/thread.html/1f638eae43fe8abcb2f8752141c96d3dbdac86a583e0790044eea727@%3Cdev.beam.apache.org%3E>
>>>>>> .
>>>>>>
>>>>>> How about this doc->website->Beam site workflow?:
>>>>>>
>>>>>> 1. The same usual newsletter [CALL FOR ITEMS] where you can
>>>>>> contribute to the google doc, with soft deadlines for when I'll publish.
>>>>>> 2. I'll publish the doc itself onto a website.
>>>>>> 3. The newsletter is mailed out in the same way, but now with a
>>>>>> shareable website link.
>>>>>> 4. We'll keep an index of archived newsletter web pages on the
>>>>>> Beam site, under the Community tab.
>>>>>> 5. If you want to submit more content after the soft deadline,
>>>>>> add it to the google doc and let me know to republish. I don't want to make
>>>>>> the publication changes automatic because that leaves us open to tampering.
>>>>>>
>>>>>>
>>>>>> This process is more laggy, so I'd suggest doing a 2 month vs monthly
>>>>>> newsletter cadence. If we're happy with this idea, I'll send in a website
>>>>>> PR for a new "Newsletter" left nav item under Community.
>>>>>>
>>>>>> Here's an example of a published newsletter: Apache Beam
>>>>>> February-March 2019
>>>>>> <https://gdoc.pub/doc/e/2PACX-1vTQIS4WkxV-HpgX5Lb6q05g4-wuIVcYd82123Mp4Y6q9fMv6Ynwd-l7dI4TrMyCrKilyU-YsoitbnZB>
>>>>>>
>>>>>>
>>>>>> - This link is permanent unless the principal google doc is
>>>>>> deleted.
>>>>>> - Changes to the google doc after web publication are not
>>>>>> automatically published on the website to protect the information integrity.
>>>>>> - Republishing is quick and easy for me if you let me know you've
>>>>>> added more.
>>>>>> - I'll improve the formatting later if we go with this route.
>>>>>>
>>>>>> Any thoughts?
>>>>>>
>>>>>> On Wed, Mar 6, 2019 at 6:13 AM Thomas Weise <th...@apache.org> wrote:
>>>>>>
>>>>>>> Similar to blog posts. A link that can be shared would also help to
>>>>>>> distribute over other channels, such as Twitter.
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Mar 6, 2019, 6:06 AM Ismaël Mejía <ie...@gmail.com> wrote:
>>>>>>>
>>>>>>>> We should have these newsletters published somewhere with a fixed
>>>>>>>> URL so we can add missing updates, I have at least missed putting stuff in
>>>>>>>> the last 3 ones just to realize when the newsletter has been already
>>>>>>>> 'published' (and sadly interesting features like zstd have not been
>>>>>>>> announced because of this).
>>>>>>>>
>>>>>>>> On Wed, Mar 6, 2019 at 2:48 PM Etienne Chauchot <
>>>>>>>> echauchot@apache.org> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>> I would add in what's been done:
>>>>>>>>>
>>>>>>>>> Work on cassandraIO (Etienne Chauchot, Mathieu Blanchard,
>>>>>>>>> Frank Shahar) : refactorings, bugfixes, new where clause, security fix
>>>>>>>>>
>>>>>>>>> Etienne
>>>>>>>>>
>>>>>>>>> Le lundi 04 mars 2019 à 18:36 +0100, Suneel Marthi a écrit :
>>>>>>>>>
>>>>>>>>> Is this the final draft? - we had 2 beam talks at Big Data Tech
>>>>>>>>> Warsaw last Wednesday - I can send the updates offline.
>>>>>>>>>
>>>>>>>>> On Mon, Mar 4, 2019 at 6:16 PM Rose Nguyen <rt...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [image: Beam.png]
>>>>>>>>>
>>>>>>>>> February-March 2019 | Newsletter
>>>>>>>>>
>>>>>>>>> What’s been done
>>>>>>>>>
>>>>>>>>> ------------------------------
>>>>>>>>>
>>>>>>>>> Apache Beam 2.10.0 released (by: many contributors)
>>>>>>>>>
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Download the release here.
>>>>>>>>> <https://beam.apache.org/get-started/downloads/>
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> See the blog post
>>>>>>>>> <https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html> for
>>>>>>>>> more details.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Apache Beam awarded the 2019 Technology of the Year Award!
>>>>>>>>>
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> InfoWorld just awarded Beam the 2019 Technology of the Year
>>>>>>>>> Award.
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> See this article
>>>>>>>>> <https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html?nsdr=true>
>>>>>>>>> for more details.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Kettle Beam 0.5 released with support for flink (by: Matt Casters)
>>>>>>>>>
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Kettle now supports Apache Flink as well as Cloud Dataflow and
>>>>>>>>> Spark.
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> See Matt’s Blog
>>>>>>>>> <http://sandbox.kettle.be/wordpress/index.php/2019/02/24/kettle-beam-update-0-5-0/>
>>>>>>>>> for more details.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> What we’re working on...
>>>>>>>>>
>>>>>>>>> ------------------------------
>>>>>>>>>
>>>>>>>>> Apache Beam 2.11.0 release (by: many contributors)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Hive Metastore Table provider for SQL (by: Anton Kedin)
>>>>>>>>>
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Support for plugging table providers through Beam SQL API to
>>>>>>>>> allow obtaining table schemas from external sources.
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> See the PR <https://github.com/apache/beam/pull/7746> for more
>>>>>>>>> details.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> User Defined Coders for the Beam Go SDK (by: Robert Burke)
>>>>>>>>>
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Working on expanding the variety of user defined types that
>>>>>>>>> can be a member of a PCollection in the Go SDK.
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> See BEAM-3306 <https://issues.apache.org/jira/browse/BEAM-3306>
>>>>>>>>> for more details.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen, Mark
>>>>>>>>> Liu, Robbe Sneyders, Juta Staes, Valentyn Tymofieiev)
>>>>>>>>>
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Beam 2.11.0 is the first release offering partial Python 3
>>>>>>>>> support.
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Many thanks to all contributors who helped to reach this
>>>>>>>>> milestone.
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> IO availablility on Python 3 is currently limited and only
>>>>>>>>> Python 3.5 version has been tested extensively.
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Stay tuned on BEAM-1251 for more details.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Notebooks for quickstarts and custom I/O (by: David Cavazos)
>>>>>>>>>
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Adding IPython notebooks and snippets
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> See [BEAM-6557] <https://github.com/apache/beam/pull/7679> for
>>>>>>>>> more details.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> New members
>>>>>>>>> ------------------------------
>>>>>>>>>
>>>>>>>>> New PMC member!
>>>>>>>>>
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Etienne Chauchot, Nantes, France
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> New Committers!
>>>>>>>>>
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Gleb Kanterov, Stockholm, Sweden
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Michael Luckey
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> New Contributors!
>>>>>>>>>
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Kyle Weaver, San Francisco, CA
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Would like to help begin implementing portability support
>>>>>>>>> for the Spark runner
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Tanay Tummapalli, Delhi, India
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Would like to contribute to Open Source this summer as part
>>>>>>>>> of Google Summer of Code
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Brian Hulette, Seattle, WA
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Contributing to Beam Portability
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Michał Walenia, Warsaw, Poland
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Working on integration and load testing
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Daniel Chen, San Francisco, CA
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Working on Beam Samza runner
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Talks & meetups
>>>>>>>>> ------------------------------
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Plugin Machine Intelligence and Apache Beam with Pentaho - Feb 7 @
>>>>>>>>> London
>>>>>>>>>
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Watch the How to Run Kettle on Apache Beam video here
>>>>>>>>> <https://skillsmatter.com/skillscasts/13405-how-to-run-kettle-on-apache-beam#video>.
>>>>>>>>>
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> See event details here
>>>>>>>>> <https://www.meetup.com/Pentaho-London-User-Group/events/256773962/>
>>>>>>>>> ..
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @ San
>>>>>>>>> Francisco, CA
>>>>>>>>>
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Organized by Thomas Weise and Austin Bennet, with speakers
>>>>>>>>> Tyler Akidau, Robert Crowe, Thomas Weise and Amar Pai
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> See event details here
>>>>>>>>> <https://www.meetup.com/San-Francisco-Apache-Beam/events/257482350/>
>>>>>>>>> and the slides for these presentation: Overview of Apache Beam
>>>>>>>>> and TensorFlow Transform (TFX) with Apache Beam
>>>>>>>>> <http://s.apache.org/beam-intro-feb-2019>, Python Streaming
>>>>>>>>> Pipelines with Beam on Flink
>>>>>>>>> <http://go.lyft.com/python-flink-beam-meetup-2019>, Dynamic
>>>>>>>>> pricing of Lyft rides using streaming
>>>>>>>>> <https://www.slideshare.net/AmarPai2/dynamic-pricing-of-lyft-rides-using-streaming>
>>>>>>>>>
>>>>>>>>> .
>>>>>>>>>
>>>>>>>>> Flink meetup - Feb 21@ Seattle, WA
>>>>>>>>>
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Speakers from Alibaba, Google, and Uber gave talks about
>>>>>>>>> Apache Flink with Hive, Tensorflow, Beam, and AthenaX.
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> See event details here
>>>>>>>>> <https://www.meetup.com/seattle-flink/events/258723322/> and
>>>>>>>>> presentations here
>>>>>>>>> <https://www.slideshare.net/BowenLi9/presentations>.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Beam Summit Europe 2019 - June 19-20 @ Berlin
>>>>>>>>>
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Beam Summit Europe 2019 will take place in Berlin on June
>>>>>>>>> 19-20.
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Speaker CfP and other details to follow soon!
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> Twitter announcement!
>>>>>>>>> <https://twitter.com/matthiasbaetens/status/1098854758893273088>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Resources
>>>>>>>>> ------------------------------
>>>>>>>>>
>>>>>>>>> Apache Jira Beginner’s Guide (by: Daniel Oliveira)
>>>>>>>>>
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> A guide
>>>>>>>>> <https://cwiki.apache.org/confluence/display/BEAM/Beam+Jira+Beginner%27s+Guide>
>>>>>>>>> to introduce Beam contributors to the basics of using the Apache Jira for
>>>>>>>>> Beam development. Feedback welcomed!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> An approach to community building from Apache Beam (by: Kenn
>>>>>>>>> Knowles)
>>>>>>>>>
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> The Apache Software Foundation has published committer
>>>>>>>>> guidelines to help Beam's community building work.
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> See the post <https://blogs.apache.org/comdev/date/20190222>
>>>>>>>>> on the ASF blog.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Exploring Beam SQL on Google Cloud Platform (by: Graham Polley)
>>>>>>>>>
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> “In this article, I’ll dive into this new feature of Beam, and
>>>>>>>>> see how it works by using a pipeline to read a data file from GCS,
>>>>>>>>> transform it, and then perform a basic calculation on the values contained
>>>>>>>>> in the file”.
>>>>>>>>> -
>>>>>>>>>
>>>>>>>>> See article
>>>>>>>>> <https://medium.com/weareservian/exploring-beam-sql-on-google-cloud-platform-b6c77f9b4af4>
>>>>>>>>> and full source code
>>>>>>>>> <https://github.com/polleyg/gcp-batch-ingestion-bigquery/blob/beam_sql/src/main/java/org/polleyg/BeamSQLMagic.java>
>>>>>>>>> .
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> *Until Next Time!*
>>>>>>>>> --
>>>>>>>>> Rose Thị Nguyễn
>>>>>>>>>
>>>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Rose Thị Nguyễn
>>>>>>
>>>>>
>
> --
> Rose Thị Nguyễn
>
--
*Aizhamal Nurmamat kyzy*
Open Source Program Manager
646-355-9740 Mobile
601 North 34th Street, Seattle, WA 98103
Re: Apache Beam Newsletter - February/March 2019
Posted by Rose Nguyen <rt...@google.com>.
Good points. With the suggested workflow I think I can support a quarterly
newsletter. I'm also happy to get more involvement from others to do this
work and we can see what cadence that allows.
On Wed, Mar 6, 2019 at 8:22 PM Kenneth Knowles <ke...@apache.org> wrote:
> Good points Melissa & Austin.
>
> - archive in the repo & on the website
> - put missed items on the next newsletter, so anyone following sees them
>
> Kenn
>
> On Wed, Mar 6, 2019 at 3:26 PM Suneel Marthi <sm...@apache.org> wrote:
>
>> I believe there was also a Beam workshop or working session in Warsaw
>> last week.
>>
>> On Wed, Mar 6, 2019 at 6:20 PM Austin Bennett <
>> whatwouldaustindo@gmail.com> wrote:
>>
>>> +1 for archive in our repo.
>>>
>>> I do follow the newsletter, but am unlikely to go back and look into the
>>> past for changes/updates.
>>>
>>> Would suggest that things that get missed in one newsletter (a concrete
>>> example, Suneel's talks not mentioned in the newsletter) would get
>>> published in the next iteration, rather than editing the past 'published'
>>> newsletter. Put another way, save editing the past for corrections (typos,
>>> things being incorrect). Else, I imagine that I'm unlikely to catch a
>>> great announcement that warranted being in the newsletter in the first
>>> place. This certainly works better with a regular/frequent release
>>> cadence, like we arrived at for version releases (then, if something misses
>>> one cut, it is not too big a deal, as the next release is coming soon).
>>>
>>>
>>>
>>>
>>> On Wed, Mar 6, 2019 at 12:50 PM Melissa Pashniak <me...@google.com>
>>> wrote:
>>>
>>>>
>>>> For step #2 (publishing onto the website), I think it would be good to
>>>> stay consistent with our existing workflows if possible. Rather than using
>>>> an external tool, what about:
>>>>
>>>> After a google doc newsletter draft is ready, convert it into a
>>>> standard markdown file and put it into our GitHub repo, perhaps in a new
>>>> newsletter directory in the website community directory [1]. These would be
>>>> listed for browsing on a Newsletters page as mentioned in step #4. People
>>>> can then just open a PR to add missing things to the pages later, and the
>>>> newsletter will be automatically updated on the website through our
>>>> standard website workflow. It also avoids the potential issue of the source
>>>> google docs disappearing in the future, as they are stored in a community
>>>> location.
>>>>
>>>> [1] https://github.com/apache/beam/tree/master/website/src/community
>>>>
>>>>
>>>> On Wed, Mar 6, 2019 at 10:36 AM Rose Nguyen <rt...@google.com>
>>>> wrote:
>>>>
>>>>> I think that would be a great idea to change formats to help with
>>>>> distribution. I'm open to suggestions! I'm currently using a Google doc to
>>>>> collect and edit, then copy/paste sending the newsletter out directly, based
>>>>> on an interpretation of this discussion
>>>>> <https://lists.apache.org/thread.html/1f638eae43fe8abcb2f8752141c96d3dbdac86a583e0790044eea727@%3Cdev.beam.apache.org%3E>
>>>>> .
>>>>>
>>>>> How about this doc->website->Beam site workflow?:
>>>>>
>>>>> 1. The same usual newsletter [CALL FOR ITEMS] where you can
>>>>> contribute to the google doc, with soft deadlines for when I'll publish.
>>>>> 2. I'll publish the doc itself onto a website.
>>>>> 3. The newsletter is mailed out in the same way, but now with a
>>>>> shareable website link.
>>>>> 4. We'll keep an index of archived newsletter web pages on the
>>>>> Beam site, under the Community tab.
>>>>> 5. If you want to submit more content after the soft deadline, add
>>>>> it to the google doc and let me know to republish. I don't want to make the
>>>>> publication changes automatic because that leaves us open to tampering.
>>>>>
>>>>>
>>>>> This process is more laggy, so I'd suggest doing a 2 month vs monthly
>>>>> newsletter cadence. If we're happy with this idea, I'll send in a website
>>>>> PR for a new "Newsletter" left nav item under Community.
>>>>>
>>>>> Here's an example of a published newsletter: Apache Beam
>>>>> February-March 2019
>>>>> <https://gdoc.pub/doc/e/2PACX-1vTQIS4WkxV-HpgX5Lb6q05g4-wuIVcYd82123Mp4Y6q9fMv6Ynwd-l7dI4TrMyCrKilyU-YsoitbnZB>
>>>>>
>>>>>
>>>>> - This link is permanent unless the principal google doc is
>>>>> deleted.
>>>>> - Changes to the google doc after web publication are not
>>>>> automatically published on the website to protect the information integrity.
>>>>> - Republishing is quick and easy for me if you let me know you've
>>>>> added more.
>>>>> - I'll improve the formatting later if we go with this route.
>>>>>
>>>>> Any thoughts?
>>>>>
>>>>> On Wed, Mar 6, 2019 at 6:13 AM Thomas Weise <th...@apache.org> wrote:
>>>>>
>>>>>> Similar to blog posts. A link that can be shared would also help to
>>>>>> distribute over other channels, such as Twitter.
>>>>>>
>>>>>>
>>>>>> On Wed, Mar 6, 2019, 6:06 AM Ismaël Mejía <ie...@gmail.com> wrote:
>>>>>>
>>>>>>> We should have these newsletters published somewhere with a fixed
>>>>>>> URL so we can add missing updates, I have at least missed putting stuff in
>>>>>>> the last 3 ones just to realize when the newsletter has been already
>>>>>>> 'published' (and sadly interesting features like zstd have not been
>>>>>>> announced because of this).
>>>>>>>
>>>>>>> On Wed, Mar 6, 2019 at 2:48 PM Etienne Chauchot <
>>>>>>> echauchot@apache.org> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>> I would add in what's been done:
>>>>>>>>
>>>>>>>> Work on cassandraIO (Etienne Chauchot, Mathieu Blanchard,
>>>>>>>> Frank Shahar) : refactorings, bugfixes, new where clause, security fix
>>>>>>>>
>>>>>>>> Etienne
>>>>>>>>
>>>>>>>> Le lundi 04 mars 2019 à 18:36 +0100, Suneel Marthi a écrit :
>>>>>>>>
>>>>>>>> Is this the final draft? - we had 2 beam talks at Big Data Tech
>>>>>>>> Warsaw last Wednesday - I can send the updates offline.
>>>>>>>>
>>>>>>>> On Mon, Mar 4, 2019 at 6:16 PM Rose Nguyen <rt...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> [image: Beam.png]
>>>>>>>>
>>>>>>>> February-March 2019 | Newsletter
>>>>>>>>
>>>>>>>> What’s been done
>>>>>>>>
>>>>>>>> ------------------------------
>>>>>>>>
>>>>>>>> Apache Beam 2.10.0 released (by: many contributors)
>>>>>>>>
>>>>>>>> -
>>>>>>>>
>>>>>>>> Download the release here.
>>>>>>>> <https://beam.apache.org/get-started/downloads/>
>>>>>>>> -
>>>>>>>>
>>>>>>>> See the blog post
>>>>>>>> <https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html> for
>>>>>>>> more details.
>>>>>>>>
>>>>>>>>
>>>>>>>> Apache Beam awarded the 2019 Technology of the Year Award!
>>>>>>>>
>>>>>>>> -
>>>>>>>>
>>>>>>>> InfoWorld just awarded Beam the 2019 Technology of the Year
>>>>>>>> Award.
>>>>>>>> -
>>>>>>>>
>>>>>>>> See this article
>>>>>>>> <https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html?nsdr=true>
>>>>>>>> for more details.
>>>>>>>>
>>>>>>>>
>>>>>>>> Kettle Beam 0.5 released with support for flink (by: Matt Casters)
>>>>>>>>
>>>>>>>> -
>>>>>>>>
>>>>>>>> Kettle now supports Apache Flink as well as Cloud Dataflow and
>>>>>>>> Spark.
>>>>>>>> -
>>>>>>>>
>>>>>>>> See Matt’s Blog
>>>>>>>> <http://sandbox.kettle.be/wordpress/index.php/2019/02/24/kettle-beam-update-0-5-0/>
>>>>>>>> for more details.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> What we’re working on...
>>>>>>>>
>>>>>>>> ------------------------------
>>>>>>>>
>>>>>>>> Apache Beam 2.11.0 release (by: many contributors)
>>>>>>>>
>>>>>>>>
>>>>>>>> Hive Metastore Table provider for SQL (by: Anton Kedin)
>>>>>>>>
>>>>>>>> -
>>>>>>>>
>>>>>>>> Support for plugging table providers through Beam SQL API to
>>>>>>>> allow obtaining table schemas from external sources.
>>>>>>>> -
>>>>>>>>
>>>>>>>> See the PR <https://github.com/apache/beam/pull/7746> for more
>>>>>>>> details.
>>>>>>>>
>>>>>>>>
>>>>>>>> User Defined Coders for the Beam Go SDK (by: Robert Burke)
>>>>>>>>
>>>>>>>> -
>>>>>>>>
>>>>>>>> Working on expanding the variety of user defined types that can
>>>>>>>> be a member of a PCollection in the Go SDK.
>>>>>>>> -
>>>>>>>>
>>>>>>>> See BEAM-3306 <https://issues.apache.org/jira/browse/BEAM-3306>
>>>>>>>> for more details.
>>>>>>>>
>>>>>>>>
>>>>>>>> Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen, Mark
>>>>>>>> Liu, Robbe Sneyders, Juta Staes, Valentyn Tymofieiev)
>>>>>>>>
>>>>>>>> -
>>>>>>>>
>>>>>>>> Beam 2.11.0 is the first release offering partial Python 3
>>>>>>>> support.
>>>>>>>> -
>>>>>>>>
>>>>>>>> Many thanks to all contributors who helped to reach this
>>>>>>>> milestone.
>>>>>>>> -
>>>>>>>>
>>>>>>>> IO availablility on Python 3 is currently limited and only
>>>>>>>> Python 3.5 version has been tested extensively.
>>>>>>>> -
>>>>>>>>
>>>>>>>> Stay tuned on BEAM-1251 for more details.
>>>>>>>>
>>>>>>>>
>>>>>>>> Notebooks for quickstarts and custom I/O (by: David Cavazos)
>>>>>>>>
>>>>>>>> -
>>>>>>>>
>>>>>>>> Adding IPython notebooks and snippets
>>>>>>>> -
>>>>>>>>
>>>>>>>> See [BEAM-6557] <https://github.com/apache/beam/pull/7679> for
>>>>>>>> more details.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> New members
>>>>>>>> ------------------------------
>>>>>>>>
>>>>>>>> New PMC member!
>>>>>>>>
>>>>>>>> -
>>>>>>>>
>>>>>>>> Etienne Chauchot, Nantes, France
>>>>>>>>
>>>>>>>>
>>>>>>>> New Committers!
>>>>>>>>
>>>>>>>> -
>>>>>>>>
>>>>>>>> Gleb Kanterov, Stockholm, Sweden
>>>>>>>> -
>>>>>>>>
>>>>>>>> Michael Luckey
>>>>>>>>
>>>>>>>>
>>>>>>>> New Contributors!
>>>>>>>>
>>>>>>>> -
>>>>>>>>
>>>>>>>> Kyle Weaver, San Francisco, CA
>>>>>>>> -
>>>>>>>>
>>>>>>>> Would like to help begin implementing portability support
>>>>>>>> for the Spark runner
>>>>>>>> -
>>>>>>>>
>>>>>>>> Tanay Tummapalli, Delhi, India
>>>>>>>> -
>>>>>>>>
>>>>>>>> Would like to contribute to Open Source this summer as part
>>>>>>>> of Google Summer of Code
>>>>>>>> -
>>>>>>>>
>>>>>>>> Brian Hulette, Seattle, WA
>>>>>>>> -
>>>>>>>>
>>>>>>>> Contributing to Beam Portability
>>>>>>>> -
>>>>>>>>
>>>>>>>> Michał Walenia, Warsaw, Poland
>>>>>>>> -
>>>>>>>>
>>>>>>>> Working on integration and load testing
>>>>>>>> -
>>>>>>>>
>>>>>>>> Daniel Chen, San Francisco, CA
>>>>>>>> -
>>>>>>>>
>>>>>>>> Working on Beam Samza runner
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Talks & meetups
>>>>>>>> ------------------------------
>>>>>>>>
>>>>>>>>
>>>>>>>> Plugin Machine Intelligence and Apache Beam with Pentaho - Feb 7 @
>>>>>>>> London
>>>>>>>>
>>>>>>>> -
>>>>>>>>
>>>>>>>> Watch the How to Run Kettle on Apache Beam video here
>>>>>>>> <https://skillsmatter.com/skillscasts/13405-how-to-run-kettle-on-apache-beam#video>.
>>>>>>>>
>>>>>>>> -
>>>>>>>>
>>>>>>>> See event details here
>>>>>>>> <https://www.meetup.com/Pentaho-London-User-Group/events/256773962/>
>>>>>>>> ..
>>>>>>>>
>>>>>>>>
>>>>>>>> Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @ San
>>>>>>>> Francisco, CA
>>>>>>>>
>>>>>>>> -
>>>>>>>>
>>>>>>>> Organized by Thomas Weise and Austin Bennet, with speakers
>>>>>>>> Tyler Akidau, Robert Crowe, Thomas Weise and Amar Pai
>>>>>>>> -
>>>>>>>>
>>>>>>>> See event details here
>>>>>>>> <https://www.meetup.com/San-Francisco-Apache-Beam/events/257482350/>
>>>>>>>> and the slides for these presentation: Overview of Apache Beam
>>>>>>>> and TensorFlow Transform (TFX) with Apache Beam
>>>>>>>> <http://s.apache.org/beam-intro-feb-2019>, Python Streaming
>>>>>>>> Pipelines with Beam on Flink
>>>>>>>> <http://go.lyft.com/python-flink-beam-meetup-2019>, Dynamic
>>>>>>>> pricing of Lyft rides using streaming
>>>>>>>> <https://www.slideshare.net/AmarPai2/dynamic-pricing-of-lyft-rides-using-streaming>
>>>>>>>>
>>>>>>>> .
>>>>>>>>
>>>>>>>> Flink meetup - Feb 21@ Seattle, WA
>>>>>>>>
>>>>>>>> -
>>>>>>>>
>>>>>>>> Speakers from Alibaba, Google, and Uber gave talks about Apache
>>>>>>>> Flink with Hive, Tensorflow, Beam, and AthenaX.
>>>>>>>> -
>>>>>>>>
>>>>>>>> See event details here
>>>>>>>> <https://www.meetup.com/seattle-flink/events/258723322/> and
>>>>>>>> presentations here
>>>>>>>> <https://www.slideshare.net/BowenLi9/presentations>.
>>>>>>>>
>>>>>>>>
>>>>>>>> Beam Summit Europe 2019 - June 19-20 @ Berlin
>>>>>>>>
>>>>>>>> -
>>>>>>>>
>>>>>>>> Beam Summit Europe 2019 will take place in Berlin on June 19-20.
>>>>>>>> -
>>>>>>>>
>>>>>>>> Speaker CfP and other details to follow soon!
>>>>>>>> -
>>>>>>>>
>>>>>>>> Twitter announcement!
>>>>>>>> <https://twitter.com/matthiasbaetens/status/1098854758893273088>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Resources
>>>>>>>> ------------------------------
>>>>>>>>
>>>>>>>> Apache Jira Beginner’s Guide (by: Daniel Oliveira)
>>>>>>>>
>>>>>>>> -
>>>>>>>>
>>>>>>>> A guide
>>>>>>>> <https://cwiki.apache.org/confluence/display/BEAM/Beam+Jira+Beginner%27s+Guide>
>>>>>>>> to introduce Beam contributors to the basics of using the Apache Jira for
>>>>>>>> Beam development. Feedback welcomed!
>>>>>>>>
>>>>>>>>
>>>>>>>> An approach to community building from Apache Beam (by: Kenn
>>>>>>>> Knowles)
>>>>>>>>
>>>>>>>> -
>>>>>>>>
>>>>>>>> The Apache Software Foundation has published committer
>>>>>>>> guidelines to help Beam's community building work.
>>>>>>>> -
>>>>>>>>
>>>>>>>> See the post <https://blogs.apache.org/comdev/date/20190222> on
>>>>>>>> the ASF blog.
>>>>>>>>
>>>>>>>>
>>>>>>>> Exploring Beam SQL on Google Cloud Platform (by: Graham Polley)
>>>>>>>>
>>>>>>>> -
>>>>>>>>
>>>>>>>> “In this article, I’ll dive into this new feature of Beam, and
>>>>>>>> see how it works by using a pipeline to read a data file from GCS,
>>>>>>>> transform it, and then perform a basic calculation on the values contained
>>>>>>>> in the file”.
>>>>>>>> -
>>>>>>>>
>>>>>>>> See article
>>>>>>>> <https://medium.com/weareservian/exploring-beam-sql-on-google-cloud-platform-b6c77f9b4af4>
>>>>>>>> and full source code
>>>>>>>> <https://github.com/polleyg/gcp-batch-ingestion-bigquery/blob/beam_sql/src/main/java/org/polleyg/BeamSQLMagic.java>
>>>>>>>> .
>>>>>>>>
>>>>>>>>
>>>>>>>> *Until Next Time!*
>>>>>>>> --
>>>>>>>> Rose Thị Nguyễn
>>>>>>>>
>>>>>>>>
>>>>>
>>>>> --
>>>>> Rose Thị Nguyễn
>>>>>
>>>>
--
Rose Thị Nguyễn
Re: Apache Beam Newsletter - February/March 2019
Posted by Kenneth Knowles <ke...@apache.org>.
Good points Melissa & Austin.
- archive in the repo & on the website
- put missed items on the next newsletter, so anyone following sees them
Kenn
On Wed, Mar 6, 2019 at 3:26 PM Suneel Marthi <sm...@apache.org> wrote:
> I believe there was also a Beam workshop or working session in Warsaw last
> week.
>
> On Wed, Mar 6, 2019 at 6:20 PM Austin Bennett <wh...@gmail.com>
> wrote:
>
>> +1 for archive in our repo.
>>
>> I do follow the newsletter, but am unlikely to go back and look into the
>> past for changes/updates.
>>
>> Would suggest that things that get missed in one newsletter (a concrete
>> example, Suneel's talks not mentioned in the newsletter) would get
>> published in the next iteration, rather than editing the past 'published'
>> newsletter. Put another way, save editing the past for corrections (typos,
>> things being incorrect). Else, I imagine that I'm unlikely to catch a
>> great announcement that warranted being in the newsletter in the first
>> place. This certainly works better with a regular/frequent release
>> cadence, like we arrived at for version releases (then, if something misses
>> one cut, it is not too big a deal, as the next release is coming soon).
>>
>>
>>
>>
>> On Wed, Mar 6, 2019 at 12:50 PM Melissa Pashniak <me...@google.com>
>> wrote:
>>
>>>
>>> For step #2 (publishing onto the website), I think it would be good to
>>> stay consistent with our existing workflows if possible. Rather than using
>>> an external tool, what about:
>>>
>>> After a google doc newsletter draft is ready, convert it into a standard
>>> markdown file and put it into our GitHub repo, perhaps in a new newsletter
>>> directory in the website community directory [1]. These would be listed for
>>> browsing on a Newsletters page as mentioned in step #4. People can then
>>> just open a PR to add missing things to the pages later, and the newsletter
>>> will be automatically updated on the website through our standard website
>>> workflow. It also avoids the potential issue of the source google docs
>>> disappearing in the future, as they are stored in a community location.
>>>
>>> [1] https://github.com/apache/beam/tree/master/website/src/community
>>>
>>>
>>> On Wed, Mar 6, 2019 at 10:36 AM Rose Nguyen <rt...@google.com> wrote:
>>>
>>>> I think that would be a great idea to change formats to help with
>>>> distribution. I'm open to suggestions! I'm currently using a Google doc to
>>>> collect and edit, then copy/paste sending the newsletter out directly, based
>>>> on an interpretation of this discussion
>>>> <https://lists.apache.org/thread.html/1f638eae43fe8abcb2f8752141c96d3dbdac86a583e0790044eea727@%3Cdev.beam.apache.org%3E>
>>>> .
>>>>
>>>> How about this doc->website->Beam site workflow?:
>>>>
>>>> 1. The same usual newsletter [CALL FOR ITEMS] where you can
>>>> contribute to the google doc, with soft deadlines for when I'll publish.
>>>> 2. I'll publish the doc itself onto a website.
>>>> 3. The newsletter is mailed out in the same way, but now with a
>>>> shareable website link.
>>>> 4. We'll keep an index of archived newsletter web pages on the Beam
>>>> site, under the Community tab.
>>>> 5. If you want to submit more content after the soft deadline, add
>>>> it to the google doc and let me know to republish. I don't want to make the
>>>> publication changes automatic because that leaves us open to tampering.
>>>>
>>>>
>>>> This process is more laggy, so I'd suggest doing a 2 month vs monthly
>>>> newsletter cadence. If we're happy with this idea, I'll send in a website
>>>> PR for a new "Newsletter" left nav item under Community.
>>>>
>>>> Here's an example of a published newsletter: Apache Beam
>>>> February-March 2019
>>>> <https://gdoc.pub/doc/e/2PACX-1vTQIS4WkxV-HpgX5Lb6q05g4-wuIVcYd82123Mp4Y6q9fMv6Ynwd-l7dI4TrMyCrKilyU-YsoitbnZB>
>>>>
>>>>
>>>> - This link is permanent unless the principal google doc is
>>>> deleted.
>>>> - Changes to the google doc after web publication are not
>>>> automatically published on the website to protect the information integrity.
>>>> - Republishing is quick and easy for me if you let me know you've
>>>> added more.
>>>> - I'll improve the formatting later if we go with this route.
>>>>
>>>> Any thoughts?
>>>>
>>>> On Wed, Mar 6, 2019 at 6:13 AM Thomas Weise <th...@apache.org> wrote:
>>>>
>>>>> Similar to blog posts. A link that can be shared would also help to
>>>>> distribute over other channels, such as Twitter.
>>>>>
>>>>>
>>>>> On Wed, Mar 6, 2019, 6:06 AM Ismaël Mejía <ie...@gmail.com> wrote:
>>>>>
>>>>>> We should have these newsletters published somewhere with a fixed URL
>>>>>> so we can add missing updates, I have at least missed putting stuff in the
>>>>>> last 3 ones just to realize when the newsletter has been already
>>>>>> 'published' (and sadly interesting features like zstd have not been
>>>>>> announced because of this).
>>>>>>
>>>>>> On Wed, Mar 6, 2019 at 2:48 PM Etienne Chauchot <ec...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>> I would add in what's been done:
>>>>>>>
>>>>>>> Work on cassandraIO (Etienne Chauchot, Mathieu Blanchard,
>>>>>>> Frank Shahar) : refactorings, bugfixes, new where clause, security fix
>>>>>>>
>>>>>>> Etienne
>>>>>>>
>>>>>>> Le lundi 04 mars 2019 à 18:36 +0100, Suneel Marthi a écrit :
>>>>>>>
>>>>>>> Is this the final draft? - we had 2 beam talks at Big Data Tech
>>>>>>> Warsaw last Wednesday - I can send the updates offline.
>>>>>>>
>>>>>>> On Mon, Mar 4, 2019 at 6:16 PM Rose Nguyen <rt...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>> [image: Beam.png]
>>>>>>>
>>>>>>> February-March 2019 | Newsletter
>>>>>>>
>>>>>>> What’s been done
>>>>>>>
>>>>>>> ------------------------------
>>>>>>>
>>>>>>> Apache Beam 2.10.0 released (by: many contributors)
>>>>>>>
>>>>>>> -
>>>>>>>
>>>>>>> Download the release here.
>>>>>>> <https://beam.apache.org/get-started/downloads/>
>>>>>>> -
>>>>>>>
>>>>>>> See the blog post
>>>>>>> <https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html> for
>>>>>>> more details.
>>>>>>>
>>>>>>>
>>>>>>> Apache Beam awarded the 2019 Technology of the Year Award!
>>>>>>>
>>>>>>> -
>>>>>>>
>>>>>>> InfoWorld just awarded Beam the 2019 Technology of the Year
>>>>>>> Award.
>>>>>>> -
>>>>>>>
>>>>>>> See this article
>>>>>>> <https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html?nsdr=true>
>>>>>>> for more details.
>>>>>>>
>>>>>>>
>>>>>>> Kettle Beam 0.5 released with support for flink (by: Matt Casters)
>>>>>>>
>>>>>>> -
>>>>>>>
>>>>>>> Kettle now supports Apache Flink as well as Cloud Dataflow and
>>>>>>> Spark.
>>>>>>> -
>>>>>>>
>>>>>>> See Matt’s Blog
>>>>>>> <http://sandbox.kettle.be/wordpress/index.php/2019/02/24/kettle-beam-update-0-5-0/>
>>>>>>> for more details.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> What we’re working on...
>>>>>>>
>>>>>>> ------------------------------
>>>>>>>
>>>>>>> Apache Beam 2.11.0 release (by: many contributors)
>>>>>>>
>>>>>>>
>>>>>>> Hive Metastore Table provider for SQL (by: Anton Kedin)
>>>>>>>
>>>>>>> -
>>>>>>>
>>>>>>> Support for plugging table providers through Beam SQL API to
>>>>>>> allow obtaining table schemas from external sources.
>>>>>>> -
>>>>>>>
>>>>>>> See the PR <https://github.com/apache/beam/pull/7746> for more
>>>>>>> details.
>>>>>>>
>>>>>>>
>>>>>>> User Defined Coders for the Beam Go SDK (by: Robert Burke)
>>>>>>>
>>>>>>> -
>>>>>>>
>>>>>>> Working on expanding the variety of user defined types that can
>>>>>>> be a member of a PCollection in the Go SDK.
>>>>>>> -
>>>>>>>
>>>>>>> See BEAM-3306 <https://issues.apache.org/jira/browse/BEAM-3306>
>>>>>>> for more details.
>>>>>>>
>>>>>>>
>>>>>>> Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen, Mark Liu,
>>>>>>> Robbe Sneyders, Juta Staes, Valentyn Tymofieiev)
>>>>>>>
>>>>>>> -
>>>>>>>
>>>>>>> Beam 2.11.0 is the first release offering partial Python 3
>>>>>>> support.
>>>>>>> -
>>>>>>>
>>>>>>> Many thanks to all contributors who helped to reach this
>>>>>>> milestone.
>>>>>>> -
>>>>>>>
>>>>>>> IO availablility on Python 3 is currently limited and only
>>>>>>> Python 3.5 version has been tested extensively.
>>>>>>> -
>>>>>>>
>>>>>>> Stay tuned on BEAM-1251 for more details.
>>>>>>>
>>>>>>>
>>>>>>> Notebooks for quickstarts and custom I/O (by: David Cavazos)
>>>>>>>
>>>>>>> -
>>>>>>>
>>>>>>> Adding IPython notebooks and snippets
>>>>>>> -
>>>>>>>
>>>>>>> See [BEAM-6557] <https://github.com/apache/beam/pull/7679> for
>>>>>>> more details.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> New members
>>>>>>> ------------------------------
>>>>>>>
>>>>>>> New PMC member!
>>>>>>>
>>>>>>> -
>>>>>>>
>>>>>>> Etienne Chauchot, Nantes, France
>>>>>>>
>>>>>>>
>>>>>>> New Committers!
>>>>>>>
>>>>>>> -
>>>>>>>
>>>>>>> Gleb Kanterov, Stockholm, Sweden
>>>>>>> -
>>>>>>>
>>>>>>> Michael Luckey
>>>>>>>
>>>>>>>
>>>>>>> New Contributors!
>>>>>>>
>>>>>>> -
>>>>>>>
>>>>>>> Kyle Weaver, San Francisco, CA
>>>>>>> -
>>>>>>>
>>>>>>> Would like to help begin implementing portability support for
>>>>>>> the Spark runner
>>>>>>> -
>>>>>>>
>>>>>>> Tanay Tummapalli, Delhi, India
>>>>>>> -
>>>>>>>
>>>>>>> Would like to contribute to Open Source this summer as part
>>>>>>> of Google Summer of Code
>>>>>>> -
>>>>>>>
>>>>>>> Brian Hulette, Seattle, WA
>>>>>>> -
>>>>>>>
>>>>>>> Contributing to Beam Portability
>>>>>>> -
>>>>>>>
>>>>>>> Michał Walenia, Warsaw, Poland
>>>>>>> -
>>>>>>>
>>>>>>> Working on integration and load testing
>>>>>>> -
>>>>>>>
>>>>>>> Daniel Chen, San Francisco, CA
>>>>>>> -
>>>>>>>
>>>>>>> Working on Beam Samza runner
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Talks & meetups
>>>>>>> ------------------------------
>>>>>>>
>>>>>>>
>>>>>>> Plugin Machine Intelligence and Apache Beam with Pentaho - Feb 7 @
>>>>>>> London
>>>>>>>
>>>>>>> -
>>>>>>>
>>>>>>> Watch the How to Run Kettle on Apache Beam video here
>>>>>>> <https://skillsmatter.com/skillscasts/13405-how-to-run-kettle-on-apache-beam#video>.
>>>>>>>
>>>>>>> -
>>>>>>>
>>>>>>> See event details here
>>>>>>> <https://www.meetup.com/Pentaho-London-User-Group/events/256773962/>
>>>>>>> ..
>>>>>>>
>>>>>>>
>>>>>>> Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @ San
>>>>>>> Francisco, CA
>>>>>>>
>>>>>>> -
>>>>>>>
>>>>>>> Organized by Thomas Weise and Austin Bennet, with speakers Tyler
>>>>>>> Akidau, Robert Crowe, Thomas Weise and Amar Pai
>>>>>>> -
>>>>>>>
>>>>>>> See event details here
>>>>>>> <https://www.meetup.com/San-Francisco-Apache-Beam/events/257482350/>
>>>>>>> and the slides for these presentation: Overview of Apache Beam
>>>>>>> and TensorFlow Transform (TFX) with Apache Beam
>>>>>>> <http://s.apache.org/beam-intro-feb-2019>, Python Streaming
>>>>>>> Pipelines with Beam on Flink
>>>>>>> <http://go.lyft.com/python-flink-beam-meetup-2019>, Dynamic
>>>>>>> pricing of Lyft rides using streaming
>>>>>>> <https://www.slideshare.net/AmarPai2/dynamic-pricing-of-lyft-rides-using-streaming>
>>>>>>>
>>>>>>> .
>>>>>>>
>>>>>>> Flink meetup - Feb 21@ Seattle, WA
>>>>>>>
>>>>>>> -
>>>>>>>
>>>>>>> Speakers from Alibaba, Google, and Uber gave talks about Apache
>>>>>>> Flink with Hive, Tensorflow, Beam, and AthenaX.
>>>>>>> -
>>>>>>>
>>>>>>> See event details here
>>>>>>> <https://www.meetup.com/seattle-flink/events/258723322/> and
>>>>>>> presentations here
>>>>>>> <https://www.slideshare.net/BowenLi9/presentations>.
>>>>>>>
>>>>>>>
>>>>>>> Beam Summit Europe 2019 - June 19-20 @ Berlin
>>>>>>>
>>>>>>> -
>>>>>>>
>>>>>>> Beam Summit Europe 2019 will take place in Berlin on June 19-20.
>>>>>>> -
>>>>>>>
>>>>>>> Speaker CfP and other details to follow soon!
>>>>>>> -
>>>>>>>
>>>>>>> Twitter announcement!
>>>>>>> <https://twitter.com/matthiasbaetens/status/1098854758893273088>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Resources
>>>>>>> ------------------------------
>>>>>>>
>>>>>>> Apache Jira Beginner’s Guide (by: Daniel Oliveira)
>>>>>>>
>>>>>>> -
>>>>>>>
>>>>>>> A guide
>>>>>>> <https://cwiki.apache.org/confluence/display/BEAM/Beam+Jira+Beginner%27s+Guide>
>>>>>>> to introduce Beam contributors to the basics of using the Apache Jira for
>>>>>>> Beam development. Feedback welcomed!
>>>>>>>
>>>>>>>
>>>>>>> An approach to community building from Apache Beam (by: Kenn
>>>>>>> Knowles)
>>>>>>>
>>>>>>> -
>>>>>>>
>>>>>>> The Apache Software Foundation has published committer
>>>>>>> guidelines to help Beam's community building work.
>>>>>>> -
>>>>>>>
>>>>>>> See the post <https://blogs.apache.org/comdev/date/20190222> on
>>>>>>> the ASF blog.
>>>>>>>
>>>>>>>
>>>>>>> Exploring Beam SQL on Google Cloud Platform (by: Graham Polley)
>>>>>>>
>>>>>>> -
>>>>>>>
>>>>>>> “In this article, I’ll dive into this new feature of Beam, and
>>>>>>> see how it works by using a pipeline to read a data file from GCS,
>>>>>>> transform it, and then perform a basic calculation on the values contained
>>>>>>> in the file”.
>>>>>>> -
>>>>>>>
>>>>>>> See article
>>>>>>> <https://medium.com/weareservian/exploring-beam-sql-on-google-cloud-platform-b6c77f9b4af4>
>>>>>>> and full source code
>>>>>>> <https://github.com/polleyg/gcp-batch-ingestion-bigquery/blob/beam_sql/src/main/java/org/polleyg/BeamSQLMagic.java>
>>>>>>> .
>>>>>>>
>>>>>>>
>>>>>>> *Until Next Time!*
>>>>>>> --
>>>>>>> Rose Thị Nguyễn
>>>>>>>
>>>>>>>
>>>>
>>>> --
>>>> Rose Thị Nguyễn
>>>>
>>>
Re: Apache Beam Newsletter - February/March 2019
Posted by Suneel Marthi <sm...@apache.org>.
I believe there was also a Beam workshop or working session in Warsaw last
week.
On Wed, Mar 6, 2019 at 6:20 PM Austin Bennett <wh...@gmail.com>
wrote:
> +1 for archive in our repo.
>
> I do follow the newsletter, but am unlikely to go back and look into the
> past for changes/updates.
>
> Would suggest that things that get missed in one newsletter (a concrete
> example, Suneel's talks not mentioned in the newsletter) would get
> published in the next iteration, rather than editing the past 'published'
> newsletter. Put another way, save editing the past for corrections (typos,
> things being incorrect). Else, I imagine that I'm unlikely to catch a
> great announcement that warranted being in the newsletter in the first
> place. This certainly works better with a regular/frequent release
> cadence, like we arrived at for version releases (then, if something misses
> one cut, it is not too big a deal, as the next release is coming soon).
>
>
>
>
> On Wed, Mar 6, 2019 at 12:50 PM Melissa Pashniak <me...@google.com>
> wrote:
>
>>
>> For step #2 (publishing onto the website), I think it would be good to
>> stay consistent with our existing workflows if possible. Rather than using
>> an external tool, what about:
>>
>> After a google doc newsletter draft is ready, convert it into a standard
>> markdown file and put it into our GitHub repo, perhaps in a new newsletter
>> directory in the website community directory [1]. These would be listed for
>> browsing on a Newsletters page as mentioned in step #4. People can then
>> just open a PR to add missing things to the pages later, and the newsletter
>> will be automatically updated on the website through our standard website
>> workflow. It also avoids the potential issue of the source google docs
>> disappearing in the future, as they are stored in a community location.
>>
>> [1] https://github.com/apache/beam/tree/master/website/src/community
>>
>>
>> On Wed, Mar 6, 2019 at 10:36 AM Rose Nguyen <rt...@google.com> wrote:
>>
>>> I think that would be a great idea to change formats to help with
>>> distribution. I'm open to suggestions! I'm currently using a Google doc to
>>> collect and edit, then copy/paste sending the newsletter out directly, based
>>> on an interpretation of this discussion
>>> <https://lists.apache.org/thread.html/1f638eae43fe8abcb2f8752141c96d3dbdac86a583e0790044eea727@%3Cdev.beam.apache.org%3E>
>>> .
>>>
>>> How about this doc->website->Beam site workflow?:
>>>
>>> 1. The same usual newsletter [CALL FOR ITEMS] where you can
>>> contribute to the google doc, with soft deadlines for when I'll publish.
>>> 2. I'll publish the doc itself onto a website.
>>> 3. The newsletter is mailed out in the same way, but now with a
>>> shareable website link.
>>> 4. We'll keep an index of archived newsletter web pages on the Beam
>>> site, under the Community tab.
>>> 5. If you want to submit more content after the soft deadline, add
>>> it to the google doc and let me know to republish. I don't want to make the
>>> publication changes automatic because that leaves us open to tampering.
>>>
>>>
>>> This process is more laggy, so I'd suggest doing a 2 month vs monthly
>>> newsletter cadence. If we're happy with this idea, I'll send in a website
>>> PR for a new "Newsletter" left nav item under Community.
>>>
>>> Here's an example of a published newsletter: Apache Beam February-March
>>> 2019
>>> <https://gdoc.pub/doc/e/2PACX-1vTQIS4WkxV-HpgX5Lb6q05g4-wuIVcYd82123Mp4Y6q9fMv6Ynwd-l7dI4TrMyCrKilyU-YsoitbnZB>
>>>
>>>
>>> - This link is permanent unless the principal google doc is deleted.
>>> - Changes to the google doc after web publication are not
>>> automatically published on the website to protect the information integrity.
>>> - Republishing is quick and easy for me if you let me know you've
>>> added more.
>>> - I'll improve the formatting later if we go with this route.
>>>
>>> Any thoughts?
>>>
>>> On Wed, Mar 6, 2019 at 6:13 AM Thomas Weise <th...@apache.org> wrote:
>>>
>>>> Similar to blog posts. A link that can be shared would also help to
>>>> distribute over other channels, such as Twitter.
>>>>
>>>>
>>>> On Wed, Mar 6, 2019, 6:06 AM Ismaël Mejía <ie...@gmail.com> wrote:
>>>>
>>>>> We should have these newsletters published somewhere with a fixed URL
>>>>> so we can add missing updates, I have at least missed putting stuff in the
>>>>> last 3 ones just to realize when the newsletter has been already
>>>>> 'published' (and sadly interesting features like zstd have not been
>>>>> announced because of this).
>>>>>
>>>>> On Wed, Mar 6, 2019 at 2:48 PM Etienne Chauchot <ec...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>> I would add in what's been done:
>>>>>>
>>>>>> Work on cassandraIO (Etienne Chauchot, Mathieu Blanchard,
>>>>>> Frank Shahar) : refactorings, bugfixes, new where clause, security fix
>>>>>>
>>>>>> Etienne
>>>>>>
>>>>>> Le lundi 04 mars 2019 à 18:36 +0100, Suneel Marthi a écrit :
>>>>>>
>>>>>> Is this the final draft? - we had 2 beam talks at Big Data Tech
>>>>>> Warsaw last Wednesday - I can send the updates offline.
>>>>>>
>>>>>> On Mon, Mar 4, 2019 at 6:16 PM Rose Nguyen <rt...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>> [image: Beam.png]
>>>>>>
>>>>>> February-March 2019 | Newsletter
>>>>>>
>>>>>> What’s been done
>>>>>>
>>>>>> ------------------------------
>>>>>>
>>>>>> Apache Beam 2.10.0 released (by: many contributors)
>>>>>>
>>>>>> -
>>>>>>
>>>>>> Download the release here.
>>>>>> <https://beam.apache.org/get-started/downloads/>
>>>>>> -
>>>>>>
>>>>>> See the blog post
>>>>>> <https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html> for
>>>>>> more details.
>>>>>>
>>>>>>
>>>>>> Apache Beam awarded the 2019 Technology of the Year Award!
>>>>>>
>>>>>> -
>>>>>>
>>>>>> InfoWorld just awarded Beam the 2019 Technology of the Year Award.
>>>>>> -
>>>>>>
>>>>>> See this article
>>>>>> <https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html?nsdr=true>
>>>>>> for more details.
>>>>>>
>>>>>>
>>>>>> Kettle Beam 0.5 released with support for flink (by: Matt Casters)
>>>>>>
>>>>>> -
>>>>>>
>>>>>> Kettle now supports Apache Flink as well as Cloud Dataflow and
>>>>>> Spark.
>>>>>> -
>>>>>>
>>>>>> See Matt’s Blog
>>>>>> <http://sandbox.kettle.be/wordpress/index.php/2019/02/24/kettle-beam-update-0-5-0/>
>>>>>> for more details.
>>>>>>
>>>>>>
>>>>>>
>>>>>> What we’re working on...
>>>>>>
>>>>>> ------------------------------
>>>>>>
>>>>>> Apache Beam 2.11.0 release (by: many contributors)
>>>>>>
>>>>>>
>>>>>> Hive Metastore Table provider for SQL (by: Anton Kedin)
>>>>>>
>>>>>> -
>>>>>>
>>>>>> Support for plugging table providers through Beam SQL API to
>>>>>> allow obtaining table schemas from external sources.
>>>>>> -
>>>>>>
>>>>>> See the PR <https://github.com/apache/beam/pull/7746> for more
>>>>>> details.
>>>>>>
>>>>>>
>>>>>> User Defined Coders for the Beam Go SDK (by: Robert Burke)
>>>>>>
>>>>>> -
>>>>>>
>>>>>> Working on expanding the variety of user defined types that can
>>>>>> be a member of a PCollection in the Go SDK.
>>>>>> -
>>>>>>
>>>>>> See BEAM-3306 <https://issues.apache.org/jira/browse/BEAM-3306>
>>>>>> for more details.
>>>>>>
>>>>>>
>>>>>> Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen, Mark Liu,
>>>>>> Robbe Sneyders, Juta Staes, Valentyn Tymofieiev)
>>>>>>
>>>>>> -
>>>>>>
>>>>>> Beam 2.11.0 is the first release offering partial Python 3
>>>>>> support.
>>>>>> -
>>>>>>
>>>>>> Many thanks to all contributors who helped to reach this
>>>>>> milestone.
>>>>>> -
>>>>>>
>>>>>> IO availablility on Python 3 is currently limited and only Python
>>>>>> 3.5 version has been tested extensively.
>>>>>> -
>>>>>>
>>>>>> Stay tuned on BEAM-1251 for more details.
>>>>>>
>>>>>>
>>>>>> Notebooks for quickstarts and custom I/O (by: David Cavazos)
>>>>>>
>>>>>> -
>>>>>>
>>>>>> Adding IPython notebooks and snippets
>>>>>> -
>>>>>>
>>>>>> See [BEAM-6557] <https://github.com/apache/beam/pull/7679> for
>>>>>> more details.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> New members
>>>>>> ------------------------------
>>>>>>
>>>>>> New PMC member!
>>>>>>
>>>>>> -
>>>>>>
>>>>>> Etienne Chauchot, Nantes, France
>>>>>>
>>>>>>
>>>>>> New Committers!
>>>>>>
>>>>>> -
>>>>>>
>>>>>> Gleb Kanterov, Stockholm, Sweden
>>>>>> -
>>>>>>
>>>>>> Michael Luckey
>>>>>>
>>>>>>
>>>>>> New Contributors!
>>>>>>
>>>>>> -
>>>>>>
>>>>>> Kyle Weaver, San Francisco, CA
>>>>>> -
>>>>>>
>>>>>> Would like to help begin implementing portability support for
>>>>>> the Spark runner
>>>>>> -
>>>>>>
>>>>>> Tanay Tummapalli, Delhi, India
>>>>>> -
>>>>>>
>>>>>> Would like to contribute to Open Source this summer as part of
>>>>>> Google Summer of Code
>>>>>> -
>>>>>>
>>>>>> Brian Hulette, Seattle, WA
>>>>>> -
>>>>>>
>>>>>> Contributing to Beam Portability
>>>>>> -
>>>>>>
>>>>>> Michał Walenia, Warsaw, Poland
>>>>>> -
>>>>>>
>>>>>> Working on integration and load testing
>>>>>> -
>>>>>>
>>>>>> Daniel Chen, San Francisco, CA
>>>>>> -
>>>>>>
>>>>>> Working on Beam Samza runner
>>>>>>
>>>>>>
>>>>>>
>>>>>> Talks & meetups
>>>>>> ------------------------------
>>>>>>
>>>>>>
>>>>>> Plugin Machine Intelligence and Apache Beam with Pentaho - Feb 7 @
>>>>>> London
>>>>>>
>>>>>> -
>>>>>>
>>>>>> Watch the How to Run Kettle on Apache Beam video here
>>>>>> <https://skillsmatter.com/skillscasts/13405-how-to-run-kettle-on-apache-beam#video>.
>>>>>>
>>>>>> -
>>>>>>
>>>>>> See event details here
>>>>>> <https://www.meetup.com/Pentaho-London-User-Group/events/256773962/>
>>>>>> ..
>>>>>>
>>>>>>
>>>>>> Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @ San
>>>>>> Francisco, CA
>>>>>>
>>>>>> -
>>>>>>
>>>>>> Organized by Thomas Weise and Austin Bennet, with speakers Tyler
>>>>>> Akidau, Robert Crowe, Thomas Weise and Amar Pai
>>>>>> -
>>>>>>
>>>>>> See event details here
>>>>>> <https://www.meetup.com/San-Francisco-Apache-Beam/events/257482350/>
>>>>>> and the slides for these presentation: Overview of Apache Beam
>>>>>> and TensorFlow Transform (TFX) with Apache Beam
>>>>>> <http://s.apache.org/beam-intro-feb-2019>, Python Streaming
>>>>>> Pipelines with Beam on Flink
>>>>>> <http://go.lyft.com/python-flink-beam-meetup-2019>, Dynamic
>>>>>> pricing of Lyft rides using streaming
>>>>>> <https://www.slideshare.net/AmarPai2/dynamic-pricing-of-lyft-rides-using-streaming>
>>>>>>
>>>>>> .
>>>>>>
>>>>>> Flink meetup - Feb 21@ Seattle, WA
>>>>>>
>>>>>> -
>>>>>>
>>>>>> Speakers from Alibaba, Google, and Uber gave talks about Apache
>>>>>> Flink with Hive, Tensorflow, Beam, and AthenaX.
>>>>>> -
>>>>>>
>>>>>> See event details here
>>>>>> <https://www.meetup.com/seattle-flink/events/258723322/> and
>>>>>> presentations here
>>>>>> <https://www.slideshare.net/BowenLi9/presentations>.
>>>>>>
>>>>>>
>>>>>> Beam Summit Europe 2019 - June 19-20 @ Berlin
>>>>>>
>>>>>> -
>>>>>>
>>>>>> Beam Summit Europe 2019 will take place in Berlin on June 19-20.
>>>>>> -
>>>>>>
>>>>>> Speaker CfP and other details to follow soon!
>>>>>> -
>>>>>>
>>>>>> Twitter announcement!
>>>>>> <https://twitter.com/matthiasbaetens/status/1098854758893273088>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Resources
>>>>>> ------------------------------
>>>>>>
>>>>>> Apache Jira Beginner’s Guide (by: Daniel Oliveira)
>>>>>>
>>>>>> -
>>>>>>
>>>>>> A guide
>>>>>> <https://cwiki.apache.org/confluence/display/BEAM/Beam+Jira+Beginner%27s+Guide>
>>>>>> to introduce Beam contributors to the basics of using the Apache Jira for
>>>>>> Beam development. Feedback welcomed!
>>>>>>
>>>>>>
>>>>>> An approach to community building from Apache Beam (by: Kenn Knowles)
>>>>>>
>>>>>> -
>>>>>>
>>>>>> The Apache Software Foundation has published committer guidelines
>>>>>> to help Beam's community building work.
>>>>>> -
>>>>>>
>>>>>> See the post <https://blogs.apache.org/comdev/date/20190222> on
>>>>>> the ASF blog.
>>>>>>
>>>>>>
>>>>>> Exploring Beam SQL on Google Cloud Platform (by: Graham Polley)
>>>>>>
>>>>>> -
>>>>>>
>>>>>> “In this article, I’ll dive into this new feature of Beam, and
>>>>>> see how it works by using a pipeline to read a data file from GCS,
>>>>>> transform it, and then perform a basic calculation on the values contained
>>>>>> in the file”.
>>>>>> -
>>>>>>
>>>>>> See article
>>>>>> <https://medium.com/weareservian/exploring-beam-sql-on-google-cloud-platform-b6c77f9b4af4>
>>>>>> and full source code
>>>>>> <https://github.com/polleyg/gcp-batch-ingestion-bigquery/blob/beam_sql/src/main/java/org/polleyg/BeamSQLMagic.java>
>>>>>> .
>>>>>>
>>>>>>
>>>>>> *Until Next Time!*
>>>>>> --
>>>>>> Rose Thị Nguyễn
>>>>>>
>>>>>>
>>>
>>> --
>>> Rose Thị Nguyễn
>>>
>>
Re: Apache Beam Newsletter - February/March 2019
Posted by Austin Bennett <wh...@gmail.com>.
+1 for archive in our repo.
I do follow the newsletter, but am unlikely to go back and look into the
past for changes/updates.
Would suggest that things that get missed in one newsletter (a concrete
example, Suneel's talks not mentioned in the newsletter) would get
published in the next iteration, rather than editing the past 'published'
newsletter. Put another way, save editing the past for corrections (typos,
things being incorrect). Else, I imagine that I'm unlikely to catch a
great announcement that warranted being in the newsletter in the first
place. This certainly works better with a regular/frequent release
cadence, like we arrived at for version releases (then, if something misses
one cut, it is not too big a deal, as the next release is coming soon).
On Wed, Mar 6, 2019 at 12:50 PM Melissa Pashniak <me...@google.com>
wrote:
>
> For step #2 (publishing onto the website), I think it would be good to
> stay consistent with our existing workflows if possible. Rather than using
> an external tool, what about:
>
> After a google doc newsletter draft is ready, convert it into a standard
> markdown file and put it into our GitHub repo, perhaps in a new newsletter
> directory in the website community directory [1]. These would be listed for
> browsing on a Newsletters page as mentioned in step #4. People can then
> just open a PR to add missing things to the pages later, and the newsletter
> will be automatically updated on the website through our standard website
> workflow. It also avoids the potential issue of the source google docs
> disappearing in the future, as they are stored in a community location.
>
> [1] https://github.com/apache/beam/tree/master/website/src/community
>
>
> On Wed, Mar 6, 2019 at 10:36 AM Rose Nguyen <rt...@google.com> wrote:
>
>> I think that would be a great idea to change formats to help with
>> distribution. I'm open to suggestions! I'm currently using a Google doc to
>> collect and edit, then copy/paste sending the newsletter out directly, based
>> on an interpretation of this discussion
>> <https://lists.apache.org/thread.html/1f638eae43fe8abcb2f8752141c96d3dbdac86a583e0790044eea727@%3Cdev.beam.apache.org%3E>
>> .
>>
>> How about this doc->website->Beam site workflow?:
>>
>> 1. The same usual newsletter [CALL FOR ITEMS] where you can
>> contribute to the google doc, with soft deadlines for when I'll publish.
>> 2. I'll publish the doc itself onto a website.
>> 3. The newsletter is mailed out in the same way, but now with a
>> shareable website link.
>> 4. We'll keep an index of archived newsletter web pages on the Beam
>> site, under the Community tab.
>> 5. If you want to submit more content after the soft deadline, add it
>> to the google doc and let me know to republish. I don't want to make the
>> publication changes automatic because that leaves us open to tampering.
>>
>>
>> This process is more laggy, so I'd suggest doing a 2 month vs monthly
>> newsletter cadence. If we're happy with this idea, I'll send in a website
>> PR for a new "Newsletter" left nav item under Community.
>>
>> Here's an example of a published newsletter: Apache Beam February-March
>> 2019
>> <https://gdoc.pub/doc/e/2PACX-1vTQIS4WkxV-HpgX5Lb6q05g4-wuIVcYd82123Mp4Y6q9fMv6Ynwd-l7dI4TrMyCrKilyU-YsoitbnZB>
>>
>>
>> - This link is permanent unless the principal google doc is deleted.
>> - Changes to the google doc after web publication are not
>> automatically published on the website to protect the information integrity.
>> - Republishing is quick and easy for me if you let me know you've
>> added more.
>> - I'll improve the formatting later if we go with this route.
>>
>> Any thoughts?
>>
>> On Wed, Mar 6, 2019 at 6:13 AM Thomas Weise <th...@apache.org> wrote:
>>
>>> Similar to blog posts. A link that can be shared would also help to
>>> distribute over other channels, such as Twitter.
>>>
>>>
>>> On Wed, Mar 6, 2019, 6:06 AM Ismaël Mejía <ie...@gmail.com> wrote:
>>>
>>>> We should have these newsletters published somewhere with a fixed URL
>>>> so we can add missing updates, I have at least missed putting stuff in the
>>>> last 3 ones just to realize when the newsletter has been already
>>>> 'published' (and sadly interesting features like zstd have not been
>>>> announced because of this).
>>>>
>>>> On Wed, Mar 6, 2019 at 2:48 PM Etienne Chauchot <ec...@apache.org>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>> I would add in what's been done:
>>>>>
>>>>> Work on cassandraIO (Etienne Chauchot, Mathieu Blanchard,
>>>>> Frank Shahar) : refactorings, bugfixes, new where clause, security fix
>>>>>
>>>>> Etienne
>>>>>
>>>>> Le lundi 04 mars 2019 à 18:36 +0100, Suneel Marthi a écrit :
>>>>>
>>>>> Is this the final draft? - we had 2 beam talks at Big Data Tech Warsaw
>>>>> last Wednesday - I can send the updates offline.
>>>>>
>>>>> On Mon, Mar 4, 2019 at 6:16 PM Rose Nguyen <rt...@google.com>
>>>>> wrote:
>>>>>
>>>>>
>>>>> [image: Beam.png]
>>>>>
>>>>> February-March 2019 | Newsletter
>>>>>
>>>>> What’s been done
>>>>>
>>>>> ------------------------------
>>>>>
>>>>> Apache Beam 2.10.0 released (by: many contributors)
>>>>>
>>>>> -
>>>>>
>>>>> Download the release here.
>>>>> <https://beam.apache.org/get-started/downloads/>
>>>>> -
>>>>>
>>>>> See the blog post
>>>>> <https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html> for
>>>>> more details.
>>>>>
>>>>>
>>>>> Apache Beam awarded the 2019 Technology of the Year Award!
>>>>>
>>>>> -
>>>>>
>>>>> InfoWorld just awarded Beam the 2019 Technology of the Year Award.
>>>>> -
>>>>>
>>>>> See this article
>>>>> <https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html?nsdr=true>
>>>>> for more details.
>>>>>
>>>>>
>>>>> Kettle Beam 0.5 released with support for flink (by: Matt Casters)
>>>>>
>>>>> -
>>>>>
>>>>> Kettle now supports Apache Flink as well as Cloud Dataflow and
>>>>> Spark.
>>>>> -
>>>>>
>>>>> See Matt’s Blog
>>>>> <http://sandbox.kettle.be/wordpress/index.php/2019/02/24/kettle-beam-update-0-5-0/>
>>>>> for more details.
>>>>>
>>>>>
>>>>>
>>>>> What we’re working on...
>>>>>
>>>>> ------------------------------
>>>>>
>>>>> Apache Beam 2.11.0 release (by: many contributors)
>>>>>
>>>>>
>>>>> Hive Metastore Table provider for SQL (by: Anton Kedin)
>>>>>
>>>>> -
>>>>>
>>>>> Support for plugging table providers through Beam SQL API to allow
>>>>> obtaining table schemas from external sources.
>>>>> -
>>>>>
>>>>> See the PR <https://github.com/apache/beam/pull/7746> for more
>>>>> details.
>>>>>
>>>>>
>>>>> User Defined Coders for the Beam Go SDK (by: Robert Burke)
>>>>>
>>>>> -
>>>>>
>>>>> Working on expanding the variety of user defined types that can be
>>>>> a member of a PCollection in the Go SDK.
>>>>> -
>>>>>
>>>>> See BEAM-3306 <https://issues.apache.org/jira/browse/BEAM-3306>
>>>>> for more details.
>>>>>
>>>>>
>>>>> Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen, Mark Liu,
>>>>> Robbe Sneyders, Juta Staes, Valentyn Tymofieiev)
>>>>>
>>>>> -
>>>>>
>>>>> Beam 2.11.0 is the first release offering partial Python 3
>>>>> support.
>>>>> -
>>>>>
>>>>> Many thanks to all contributors who helped to reach this milestone.
>>>>> -
>>>>>
>>>>> IO availablility on Python 3 is currently limited and only Python
>>>>> 3.5 version has been tested extensively.
>>>>> -
>>>>>
>>>>> Stay tuned on BEAM-1251 for more details.
>>>>>
>>>>>
>>>>> Notebooks for quickstarts and custom I/O (by: David Cavazos)
>>>>>
>>>>> -
>>>>>
>>>>> Adding IPython notebooks and snippets
>>>>> -
>>>>>
>>>>> See [BEAM-6557] <https://github.com/apache/beam/pull/7679> for
>>>>> more details.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> New members
>>>>> ------------------------------
>>>>>
>>>>> New PMC member!
>>>>>
>>>>> -
>>>>>
>>>>> Etienne Chauchot, Nantes, France
>>>>>
>>>>>
>>>>> New Committers!
>>>>>
>>>>> -
>>>>>
>>>>> Gleb Kanterov, Stockholm, Sweden
>>>>> -
>>>>>
>>>>> Michael Luckey
>>>>>
>>>>>
>>>>> New Contributors!
>>>>>
>>>>> -
>>>>>
>>>>> Kyle Weaver, San Francisco, CA
>>>>> -
>>>>>
>>>>> Would like to help begin implementing portability support for
>>>>> the Spark runner
>>>>> -
>>>>>
>>>>> Tanay Tummapalli, Delhi, India
>>>>> -
>>>>>
>>>>> Would like to contribute to Open Source this summer as part of
>>>>> Google Summer of Code
>>>>> -
>>>>>
>>>>> Brian Hulette, Seattle, WA
>>>>> -
>>>>>
>>>>> Contributing to Beam Portability
>>>>> -
>>>>>
>>>>> Michał Walenia, Warsaw, Poland
>>>>> -
>>>>>
>>>>> Working on integration and load testing
>>>>> -
>>>>>
>>>>> Daniel Chen, San Francisco, CA
>>>>> -
>>>>>
>>>>> Working on Beam Samza runner
>>>>>
>>>>>
>>>>>
>>>>> Talks & meetups
>>>>> ------------------------------
>>>>>
>>>>>
>>>>> Plugin Machine Intelligence and Apache Beam with Pentaho - Feb 7 @
>>>>> London
>>>>>
>>>>> -
>>>>>
>>>>> Watch the How to Run Kettle on Apache Beam video here
>>>>> <https://skillsmatter.com/skillscasts/13405-how-to-run-kettle-on-apache-beam#video>.
>>>>>
>>>>> -
>>>>>
>>>>> See event details here
>>>>> <https://www.meetup.com/Pentaho-London-User-Group/events/256773962/>
>>>>> ..
>>>>>
>>>>>
>>>>> Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @ San
>>>>> Francisco, CA
>>>>>
>>>>> -
>>>>>
>>>>> Organized by Thomas Weise and Austin Bennet, with speakers Tyler
>>>>> Akidau, Robert Crowe, Thomas Weise and Amar Pai
>>>>> -
>>>>>
>>>>> See event details here
>>>>> <https://www.meetup.com/San-Francisco-Apache-Beam/events/257482350/>
>>>>> and the slides for these presentation: Overview of Apache Beam and
>>>>> TensorFlow Transform (TFX) with Apache Beam
>>>>> <http://s.apache.org/beam-intro-feb-2019>, Python Streaming
>>>>> Pipelines with Beam on Flink
>>>>> <http://go.lyft.com/python-flink-beam-meetup-2019>, Dynamic
>>>>> pricing of Lyft rides using streaming
>>>>> <https://www.slideshare.net/AmarPai2/dynamic-pricing-of-lyft-rides-using-streaming>
>>>>>
>>>>> .
>>>>>
>>>>> Flink meetup - Feb 21@ Seattle, WA
>>>>>
>>>>> -
>>>>>
>>>>> Speakers from Alibaba, Google, and Uber gave talks about Apache
>>>>> Flink with Hive, Tensorflow, Beam, and AthenaX.
>>>>> -
>>>>>
>>>>> See event details here
>>>>> <https://www.meetup.com/seattle-flink/events/258723322/> and
>>>>> presentations here
>>>>> <https://www.slideshare.net/BowenLi9/presentations>.
>>>>>
>>>>>
>>>>> Beam Summit Europe 2019 - June 19-20 @ Berlin
>>>>>
>>>>> -
>>>>>
>>>>> Beam Summit Europe 2019 will take place in Berlin on June 19-20.
>>>>> -
>>>>>
>>>>> Speaker CfP and other details to follow soon!
>>>>> -
>>>>>
>>>>> Twitter announcement!
>>>>> <https://twitter.com/matthiasbaetens/status/1098854758893273088>
>>>>>
>>>>>
>>>>>
>>>>> Resources
>>>>> ------------------------------
>>>>>
>>>>> Apache Jira Beginner’s Guide (by: Daniel Oliveira)
>>>>>
>>>>> -
>>>>>
>>>>> A guide
>>>>> <https://cwiki.apache.org/confluence/display/BEAM/Beam+Jira+Beginner%27s+Guide>
>>>>> to introduce Beam contributors to the basics of using the Apache Jira for
>>>>> Beam development. Feedback welcomed!
>>>>>
>>>>>
>>>>> An approach to community building from Apache Beam (by: Kenn Knowles)
>>>>>
>>>>> -
>>>>>
>>>>> The Apache Software Foundation has published committer guidelines
>>>>> to help Beam's community building work.
>>>>> -
>>>>>
>>>>> See the post <https://blogs.apache.org/comdev/date/20190222> on
>>>>> the ASF blog.
>>>>>
>>>>>
>>>>> Exploring Beam SQL on Google Cloud Platform (by: Graham Polley)
>>>>>
>>>>> -
>>>>>
>>>>> “In this article, I’ll dive into this new feature of Beam, and see
>>>>> how it works by using a pipeline to read a data file from GCS, transform
>>>>> it, and then perform a basic calculation on the values contained in the
>>>>> file”.
>>>>> -
>>>>>
>>>>> See article
>>>>> <https://medium.com/weareservian/exploring-beam-sql-on-google-cloud-platform-b6c77f9b4af4>
>>>>> and full source code
>>>>> <https://github.com/polleyg/gcp-batch-ingestion-bigquery/blob/beam_sql/src/main/java/org/polleyg/BeamSQLMagic.java>
>>>>> .
>>>>>
>>>>>
>>>>> *Until Next Time!*
>>>>> --
>>>>> Rose Thị Nguyễn
>>>>>
>>>>>
>>
>> --
>> Rose Thị Nguyễn
>>
>
Re: Apache Beam Newsletter - February/March 2019
Posted by Melissa Pashniak <me...@google.com>.
For step #2 (publishing onto the website), I think it would be good to stay
consistent with our existing workflows if possible. Rather than using an
external tool, what about:
After a google doc newsletter draft is ready, convert it into a standard
markdown file and put it into our GitHub repo, perhaps in a new newsletter
directory in the website community directory [1]. These would be listed for
browsing on a Newsletters page as mentioned in step #4. People can then
just open a PR to add missing things to the pages later, and the newsletter
will be automatically updated on the website through our standard website
workflow. It also avoids the potential issue of the source google docs
disappearing in the future, as they are stored in a community location.
[1] https://github.com/apache/beam/tree/master/website/src/community
On Wed, Mar 6, 2019 at 10:36 AM Rose Nguyen <rt...@google.com> wrote:
> I think that would be a great idea to change formats to help with
> distribution. I'm open to suggestions! I'm currently using a Google doc to
> collect and edit, then copy/paste sending the newsletter out directly, based
> on an interpretation of this discussion
> <https://lists.apache.org/thread.html/1f638eae43fe8abcb2f8752141c96d3dbdac86a583e0790044eea727@%3Cdev.beam.apache.org%3E>
> .
>
> How about this doc->website->Beam site workflow?:
>
> 1. The same usual newsletter [CALL FOR ITEMS] where you can contribute
> to the google doc, with soft deadlines for when I'll publish.
> 2. I'll publish the doc itself onto a website.
> 3. The newsletter is mailed out in the same way, but now with a
> shareable website link.
> 4. We'll keep an index of archived newsletter web pages on the Beam
> site, under the Community tab.
> 5. If you want to submit more content after the soft deadline, add it
> to the google doc and let me know to republish. I don't want to make the
> publication changes automatic because that leaves us open to tampering.
>
>
> This process is more laggy, so I'd suggest doing a 2 month vs monthly
> newsletter cadence. If we're happy with this idea, I'll send in a website
> PR for a new "Newsletter" left nav item under Community.
>
> Here's an example of a published newsletter: Apache Beam February-March
> 2019
> <https://gdoc.pub/doc/e/2PACX-1vTQIS4WkxV-HpgX5Lb6q05g4-wuIVcYd82123Mp4Y6q9fMv6Ynwd-l7dI4TrMyCrKilyU-YsoitbnZB>
>
>
> - This link is permanent unless the principal google doc is deleted.
> - Changes to the google doc after web publication are not
> automatically published on the website to protect the information integrity.
> - Republishing is quick and easy for me if you let me know you've
> added more.
> - I'll improve the formatting later if we go with this route.
>
> Any thoughts?
>
> On Wed, Mar 6, 2019 at 6:13 AM Thomas Weise <th...@apache.org> wrote:
>
>> Similar to blog posts. A link that can be shared would also help to
>> distribute over other channels, such as Twitter.
>>
>>
>> On Wed, Mar 6, 2019, 6:06 AM Ismaël Mejía <ie...@gmail.com> wrote:
>>
>>> We should have these newsletters published somewhere with a fixed URL so
>>> we can add missing updates, I have at least missed putting stuff in the
>>> last 3 ones just to realize when the newsletter has been already
>>> 'published' (and sadly interesting features like zstd have not been
>>> announced because of this).
>>>
>>> On Wed, Mar 6, 2019 at 2:48 PM Etienne Chauchot <ec...@apache.org>
>>> wrote:
>>>
>>>> Hi,
>>>> I would add in what's been done:
>>>>
>>>> Work on cassandraIO (Etienne Chauchot, Mathieu Blanchard, Frank Shahar)
>>>> : refactorings, bugfixes, new where clause, security fix
>>>>
>>>> Etienne
>>>>
>>>> Le lundi 04 mars 2019 à 18:36 +0100, Suneel Marthi a écrit :
>>>>
>>>> Is this the final draft? - we had 2 beam talks at Big Data Tech Warsaw
>>>> last Wednesday - I can send the updates offline.
>>>>
>>>> On Mon, Mar 4, 2019 at 6:16 PM Rose Nguyen <rt...@google.com> wrote:
>>>>
>>>>
>>>> [image: Beam.png]
>>>>
>>>> February-March 2019 | Newsletter
>>>>
>>>> What’s been done
>>>>
>>>> ------------------------------
>>>>
>>>> Apache Beam 2.10.0 released (by: many contributors)
>>>>
>>>> -
>>>>
>>>> Download the release here.
>>>> <https://beam.apache.org/get-started/downloads/>
>>>> -
>>>>
>>>> See the blog post
>>>> <https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html> for more
>>>> details.
>>>>
>>>>
>>>> Apache Beam awarded the 2019 Technology of the Year Award!
>>>>
>>>> -
>>>>
>>>> InfoWorld just awarded Beam the 2019 Technology of the Year Award.
>>>> -
>>>>
>>>> See this article
>>>> <https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html?nsdr=true>
>>>> for more details.
>>>>
>>>>
>>>> Kettle Beam 0.5 released with support for flink (by: Matt Casters)
>>>>
>>>> -
>>>>
>>>> Kettle now supports Apache Flink as well as Cloud Dataflow and
>>>> Spark.
>>>> -
>>>>
>>>> See Matt’s Blog
>>>> <http://sandbox.kettle.be/wordpress/index.php/2019/02/24/kettle-beam-update-0-5-0/>
>>>> for more details.
>>>>
>>>>
>>>>
>>>> What we’re working on...
>>>>
>>>> ------------------------------
>>>>
>>>> Apache Beam 2.11.0 release (by: many contributors)
>>>>
>>>>
>>>> Hive Metastore Table provider for SQL (by: Anton Kedin)
>>>>
>>>> -
>>>>
>>>> Support for plugging table providers through Beam SQL API to allow
>>>> obtaining table schemas from external sources.
>>>> -
>>>>
>>>> See the PR <https://github.com/apache/beam/pull/7746> for more
>>>> details.
>>>>
>>>>
>>>> User Defined Coders for the Beam Go SDK (by: Robert Burke)
>>>>
>>>> -
>>>>
>>>> Working on expanding the variety of user defined types that can be
>>>> a member of a PCollection in the Go SDK.
>>>> -
>>>>
>>>> See BEAM-3306 <https://issues.apache.org/jira/browse/BEAM-3306> for
>>>> more details.
>>>>
>>>>
>>>> Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen, Mark Liu,
>>>> Robbe Sneyders, Juta Staes, Valentyn Tymofieiev)
>>>>
>>>> -
>>>>
>>>> Beam 2.11.0 is the first release offering partial Python 3 support.
>>>> -
>>>>
>>>> Many thanks to all contributors who helped to reach this milestone.
>>>> -
>>>>
>>>> IO availablility on Python 3 is currently limited and only Python
>>>> 3.5 version has been tested extensively.
>>>> -
>>>>
>>>> Stay tuned on BEAM-1251 for more details.
>>>>
>>>>
>>>> Notebooks for quickstarts and custom I/O (by: David Cavazos)
>>>>
>>>> -
>>>>
>>>> Adding IPython notebooks and snippets
>>>> -
>>>>
>>>> See [BEAM-6557] <https://github.com/apache/beam/pull/7679> for more
>>>> details.
>>>>
>>>>
>>>>
>>>>
>>>> New members
>>>> ------------------------------
>>>>
>>>> New PMC member!
>>>>
>>>> -
>>>>
>>>> Etienne Chauchot, Nantes, France
>>>>
>>>>
>>>> New Committers!
>>>>
>>>> -
>>>>
>>>> Gleb Kanterov, Stockholm, Sweden
>>>> -
>>>>
>>>> Michael Luckey
>>>>
>>>>
>>>> New Contributors!
>>>>
>>>> -
>>>>
>>>> Kyle Weaver, San Francisco, CA
>>>> -
>>>>
>>>> Would like to help begin implementing portability support for
>>>> the Spark runner
>>>> -
>>>>
>>>> Tanay Tummapalli, Delhi, India
>>>> -
>>>>
>>>> Would like to contribute to Open Source this summer as part of
>>>> Google Summer of Code
>>>> -
>>>>
>>>> Brian Hulette, Seattle, WA
>>>> -
>>>>
>>>> Contributing to Beam Portability
>>>> -
>>>>
>>>> Michał Walenia, Warsaw, Poland
>>>> -
>>>>
>>>> Working on integration and load testing
>>>> -
>>>>
>>>> Daniel Chen, San Francisco, CA
>>>> -
>>>>
>>>> Working on Beam Samza runner
>>>>
>>>>
>>>>
>>>> Talks & meetups
>>>> ------------------------------
>>>>
>>>>
>>>> Plugin Machine Intelligence and Apache Beam with Pentaho - Feb 7 @
>>>> London
>>>>
>>>> -
>>>>
>>>> Watch the How to Run Kettle on Apache Beam video here
>>>> <https://skillsmatter.com/skillscasts/13405-how-to-run-kettle-on-apache-beam#video>.
>>>>
>>>> -
>>>>
>>>> See event details here
>>>> <https://www.meetup.com/Pentaho-London-User-Group/events/256773962/>
>>>> ..
>>>>
>>>>
>>>> Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @ San
>>>> Francisco, CA
>>>>
>>>> -
>>>>
>>>> Organized by Thomas Weise and Austin Bennet, with speakers Tyler
>>>> Akidau, Robert Crowe, Thomas Weise and Amar Pai
>>>> -
>>>>
>>>> See event details here
>>>> <https://www.meetup.com/San-Francisco-Apache-Beam/events/257482350/>
>>>> and the slides for these presentation: Overview of Apache Beam and
>>>> TensorFlow Transform (TFX) with Apache Beam
>>>> <http://s.apache.org/beam-intro-feb-2019>, Python Streaming
>>>> Pipelines with Beam on Flink
>>>> <http://go.lyft.com/python-flink-beam-meetup-2019>, Dynamic pricing
>>>> of Lyft rides using streaming
>>>> <https://www.slideshare.net/AmarPai2/dynamic-pricing-of-lyft-rides-using-streaming>
>>>>
>>>> .
>>>>
>>>> Flink meetup - Feb 21@ Seattle, WA
>>>>
>>>> -
>>>>
>>>> Speakers from Alibaba, Google, and Uber gave talks about Apache
>>>> Flink with Hive, Tensorflow, Beam, and AthenaX.
>>>> -
>>>>
>>>> See event details here
>>>> <https://www.meetup.com/seattle-flink/events/258723322/> and
>>>> presentations here
>>>> <https://www.slideshare.net/BowenLi9/presentations>.
>>>>
>>>>
>>>> Beam Summit Europe 2019 - June 19-20 @ Berlin
>>>>
>>>> -
>>>>
>>>> Beam Summit Europe 2019 will take place in Berlin on June 19-20.
>>>> -
>>>>
>>>> Speaker CfP and other details to follow soon!
>>>> -
>>>>
>>>> Twitter announcement!
>>>> <https://twitter.com/matthiasbaetens/status/1098854758893273088>
>>>>
>>>>
>>>>
>>>> Resources
>>>> ------------------------------
>>>>
>>>> Apache Jira Beginner’s Guide (by: Daniel Oliveira)
>>>>
>>>> -
>>>>
>>>> A guide
>>>> <https://cwiki.apache.org/confluence/display/BEAM/Beam+Jira+Beginner%27s+Guide>
>>>> to introduce Beam contributors to the basics of using the Apache Jira for
>>>> Beam development. Feedback welcomed!
>>>>
>>>>
>>>> An approach to community building from Apache Beam (by: Kenn Knowles)
>>>>
>>>> -
>>>>
>>>> The Apache Software Foundation has published committer guidelines
>>>> to help Beam's community building work.
>>>> -
>>>>
>>>> See the post <https://blogs.apache.org/comdev/date/20190222> on the
>>>> ASF blog.
>>>>
>>>>
>>>> Exploring Beam SQL on Google Cloud Platform (by: Graham Polley)
>>>>
>>>> -
>>>>
>>>> “In this article, I’ll dive into this new feature of Beam, and see
>>>> how it works by using a pipeline to read a data file from GCS, transform
>>>> it, and then perform a basic calculation on the values contained in the
>>>> file”.
>>>> -
>>>>
>>>> See article
>>>> <https://medium.com/weareservian/exploring-beam-sql-on-google-cloud-platform-b6c77f9b4af4>
>>>> and full source code
>>>> <https://github.com/polleyg/gcp-batch-ingestion-bigquery/blob/beam_sql/src/main/java/org/polleyg/BeamSQLMagic.java>
>>>> .
>>>>
>>>>
>>>> *Until Next Time!*
>>>> --
>>>> Rose Thị Nguyễn
>>>>
>>>>
>
> --
> Rose Thị Nguyễn
>
Re: Apache Beam Newsletter - February/March 2019
Posted by Rose Nguyen <rt...@google.com>.
I think that would be a great idea to change formats to help with
distribution. I'm open to suggestions! I'm currently using a Google doc to
collect and edit, then copy/paste sending the newsletter out directly, based
on an interpretation of this discussion
<https://lists.apache.org/thread.html/1f638eae43fe8abcb2f8752141c96d3dbdac86a583e0790044eea727@%3Cdev.beam.apache.org%3E>
.
How about this doc->website->Beam site workflow?:
1. The same usual newsletter [CALL FOR ITEMS] where you can contribute
to the google doc, with soft deadlines for when I'll publish.
2. I'll publish the doc itself onto a website.
3. The newsletter is mailed out in the same way, but now with a
shareable website link.
4. We'll keep an index of archived newsletter web pages on the Beam
site, under the Community tab.
5. If you want to submit more content after the soft deadline, add it to
the google doc and let me know to republish. I don't want to make the
publication changes automatic because that leaves us open to tampering.
This process is more laggy, so I'd suggest doing a 2 month vs monthly
newsletter cadence. If we're happy with this idea, I'll send in a website
PR for a new "Newsletter" left nav item under Community.
Here's an example of a published newsletter: Apache Beam February-March 2019
<https://gdoc.pub/doc/e/2PACX-1vTQIS4WkxV-HpgX5Lb6q05g4-wuIVcYd82123Mp4Y6q9fMv6Ynwd-l7dI4TrMyCrKilyU-YsoitbnZB>
- This link is permanent unless the principal google doc is deleted.
- Changes to the google doc after web publication are not automatically
published on the website to protect the information integrity.
- Republishing is quick and easy for me if you let me know you've added
more.
- I'll improve the formatting later if we go with this route.
Any thoughts?
On Wed, Mar 6, 2019 at 6:13 AM Thomas Weise <th...@apache.org> wrote:
> Similar to blog posts. A link that can be shared would also help to
> distribute over other channels, such as Twitter.
>
>
> On Wed, Mar 6, 2019, 6:06 AM Ismaël Mejía <ie...@gmail.com> wrote:
>
>> We should have these newsletters published somewhere with a fixed URL so
>> we can add missing updates, I have at least missed putting stuff in the
>> last 3 ones just to realize when the newsletter has been already
>> 'published' (and sadly interesting features like zstd have not been
>> announced because of this).
>>
>> On Wed, Mar 6, 2019 at 2:48 PM Etienne Chauchot <ec...@apache.org>
>> wrote:
>>
>>> Hi,
>>> I would add in what's been done:
>>>
>>> Work on cassandraIO (Etienne Chauchot, Mathieu Blanchard, Frank Shahar)
>>> : refactorings, bugfixes, new where clause, security fix
>>>
>>> Etienne
>>>
>>> Le lundi 04 mars 2019 à 18:36 +0100, Suneel Marthi a écrit :
>>>
>>> Is this the final draft? - we had 2 beam talks at Big Data Tech Warsaw
>>> last Wednesday - I can send the updates offline.
>>>
>>> On Mon, Mar 4, 2019 at 6:16 PM Rose Nguyen <rt...@google.com> wrote:
>>>
>>>
>>> [image: Beam.png]
>>>
>>> February-March 2019 | Newsletter
>>>
>>> What’s been done
>>>
>>> ------------------------------
>>>
>>> Apache Beam 2.10.0 released (by: many contributors)
>>>
>>> -
>>>
>>> Download the release here.
>>> <https://beam.apache.org/get-started/downloads/>
>>> -
>>>
>>> See the blog post
>>> <https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html> for more
>>> details.
>>>
>>>
>>> Apache Beam awarded the 2019 Technology of the Year Award!
>>>
>>> -
>>>
>>> InfoWorld just awarded Beam the 2019 Technology of the Year Award.
>>> -
>>>
>>> See this article
>>> <https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html?nsdr=true>
>>> for more details.
>>>
>>>
>>> Kettle Beam 0.5 released with support for flink (by: Matt Casters)
>>>
>>> -
>>>
>>> Kettle now supports Apache Flink as well as Cloud Dataflow and Spark.
>>> -
>>>
>>> See Matt’s Blog
>>> <http://sandbox.kettle.be/wordpress/index.php/2019/02/24/kettle-beam-update-0-5-0/>
>>> for more details.
>>>
>>>
>>>
>>> What we’re working on...
>>>
>>> ------------------------------
>>>
>>> Apache Beam 2.11.0 release (by: many contributors)
>>>
>>>
>>> Hive Metastore Table provider for SQL (by: Anton Kedin)
>>>
>>> -
>>>
>>> Support for plugging table providers through Beam SQL API to allow
>>> obtaining table schemas from external sources.
>>> -
>>>
>>> See the PR <https://github.com/apache/beam/pull/7746> for more
>>> details.
>>>
>>>
>>> User Defined Coders for the Beam Go SDK (by: Robert Burke)
>>>
>>> -
>>>
>>> Working on expanding the variety of user defined types that can be a
>>> member of a PCollection in the Go SDK.
>>> -
>>>
>>> See BEAM-3306 <https://issues.apache.org/jira/browse/BEAM-3306> for
>>> more details.
>>>
>>>
>>> Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen, Mark Liu,
>>> Robbe Sneyders, Juta Staes, Valentyn Tymofieiev)
>>>
>>> -
>>>
>>> Beam 2.11.0 is the first release offering partial Python 3 support.
>>> -
>>>
>>> Many thanks to all contributors who helped to reach this milestone.
>>> -
>>>
>>> IO availablility on Python 3 is currently limited and only Python
>>> 3.5 version has been tested extensively.
>>> -
>>>
>>> Stay tuned on BEAM-1251 for more details.
>>>
>>>
>>> Notebooks for quickstarts and custom I/O (by: David Cavazos)
>>>
>>> -
>>>
>>> Adding IPython notebooks and snippets
>>> -
>>>
>>> See [BEAM-6557] <https://github.com/apache/beam/pull/7679> for more
>>> details.
>>>
>>>
>>>
>>>
>>> New members
>>> ------------------------------
>>>
>>> New PMC member!
>>>
>>> -
>>>
>>> Etienne Chauchot, Nantes, France
>>>
>>>
>>> New Committers!
>>>
>>> -
>>>
>>> Gleb Kanterov, Stockholm, Sweden
>>> -
>>>
>>> Michael Luckey
>>>
>>>
>>> New Contributors!
>>>
>>> -
>>>
>>> Kyle Weaver, San Francisco, CA
>>> -
>>>
>>> Would like to help begin implementing portability support for the
>>> Spark runner
>>> -
>>>
>>> Tanay Tummapalli, Delhi, India
>>> -
>>>
>>> Would like to contribute to Open Source this summer as part of
>>> Google Summer of Code
>>> -
>>>
>>> Brian Hulette, Seattle, WA
>>> -
>>>
>>> Contributing to Beam Portability
>>> -
>>>
>>> Michał Walenia, Warsaw, Poland
>>> -
>>>
>>> Working on integration and load testing
>>> -
>>>
>>> Daniel Chen, San Francisco, CA
>>> -
>>>
>>> Working on Beam Samza runner
>>>
>>>
>>>
>>> Talks & meetups
>>> ------------------------------
>>>
>>>
>>> Plugin Machine Intelligence and Apache Beam with Pentaho - Feb 7 @ London
>>>
>>> -
>>>
>>> Watch the How to Run Kettle on Apache Beam video here
>>> <https://skillsmatter.com/skillscasts/13405-how-to-run-kettle-on-apache-beam#video>.
>>>
>>> -
>>>
>>> See event details here
>>> <https://www.meetup.com/Pentaho-London-User-Group/events/256773962/>
>>> ..
>>>
>>>
>>> Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @ San
>>> Francisco, CA
>>>
>>> -
>>>
>>> Organized by Thomas Weise and Austin Bennet, with speakers Tyler
>>> Akidau, Robert Crowe, Thomas Weise and Amar Pai
>>> -
>>>
>>> See event details here
>>> <https://www.meetup.com/San-Francisco-Apache-Beam/events/257482350/>
>>> and the slides for these presentation: Overview of Apache Beam and
>>> TensorFlow Transform (TFX) with Apache Beam
>>> <http://s.apache.org/beam-intro-feb-2019>, Python Streaming
>>> Pipelines with Beam on Flink
>>> <http://go.lyft.com/python-flink-beam-meetup-2019>, Dynamic pricing
>>> of Lyft rides using streaming
>>> <https://www.slideshare.net/AmarPai2/dynamic-pricing-of-lyft-rides-using-streaming>
>>>
>>> .
>>>
>>> Flink meetup - Feb 21@ Seattle, WA
>>>
>>> -
>>>
>>> Speakers from Alibaba, Google, and Uber gave talks about Apache
>>> Flink with Hive, Tensorflow, Beam, and AthenaX.
>>> -
>>>
>>> See event details here
>>> <https://www.meetup.com/seattle-flink/events/258723322/> and
>>> presentations here
>>> <https://www.slideshare.net/BowenLi9/presentations>.
>>>
>>>
>>> Beam Summit Europe 2019 - June 19-20 @ Berlin
>>>
>>> -
>>>
>>> Beam Summit Europe 2019 will take place in Berlin on June 19-20.
>>> -
>>>
>>> Speaker CfP and other details to follow soon!
>>> -
>>>
>>> Twitter announcement!
>>> <https://twitter.com/matthiasbaetens/status/1098854758893273088>
>>>
>>>
>>>
>>> Resources
>>> ------------------------------
>>>
>>> Apache Jira Beginner’s Guide (by: Daniel Oliveira)
>>>
>>> -
>>>
>>> A guide
>>> <https://cwiki.apache.org/confluence/display/BEAM/Beam+Jira+Beginner%27s+Guide>
>>> to introduce Beam contributors to the basics of using the Apache Jira for
>>> Beam development. Feedback welcomed!
>>>
>>>
>>> An approach to community building from Apache Beam (by: Kenn Knowles)
>>>
>>> -
>>>
>>> The Apache Software Foundation has published committer guidelines to
>>> help Beam's community building work.
>>> -
>>>
>>> See the post <https://blogs.apache.org/comdev/date/20190222> on the
>>> ASF blog.
>>>
>>>
>>> Exploring Beam SQL on Google Cloud Platform (by: Graham Polley)
>>>
>>> -
>>>
>>> “In this article, I’ll dive into this new feature of Beam, and see
>>> how it works by using a pipeline to read a data file from GCS, transform
>>> it, and then perform a basic calculation on the values contained in the
>>> file”.
>>> -
>>>
>>> See article
>>> <https://medium.com/weareservian/exploring-beam-sql-on-google-cloud-platform-b6c77f9b4af4>
>>> and full source code
>>> <https://github.com/polleyg/gcp-batch-ingestion-bigquery/blob/beam_sql/src/main/java/org/polleyg/BeamSQLMagic.java>
>>> .
>>>
>>>
>>> *Until Next Time!*
>>> --
>>> Rose Thị Nguyễn
>>>
>>>
--
Rose Thị Nguyễn
Re: Apache Beam Newsletter - February/March 2019
Posted by Thomas Weise <th...@apache.org>.
Similar to blog posts. A link that can be shared would also help to
distribute over other channels, such as Twitter.
On Wed, Mar 6, 2019, 6:06 AM Ismaël Mejía <ie...@gmail.com> wrote:
> We should have these newsletters published somewhere with a fixed URL so
> we can add missing updates, I have at least missed putting stuff in the
> last 3 ones just to realize when the newsletter has been already
> 'published' (and sadly interesting features like zstd have not been
> announced because of this).
>
> On Wed, Mar 6, 2019 at 2:48 PM Etienne Chauchot <ec...@apache.org>
> wrote:
>
>> Hi,
>> I would add in what's been done:
>>
>> Work on cassandraIO (Etienne Chauchot, Mathieu Blanchard, Frank Shahar) :
>> refactorings, bugfixes, new where clause, security fix
>>
>> Etienne
>>
>> Le lundi 04 mars 2019 à 18:36 +0100, Suneel Marthi a écrit :
>>
>> Is this the final draft? - we had 2 beam talks at Big Data Tech Warsaw
>> last Wednesday - I can send the updates offline.
>>
>> On Mon, Mar 4, 2019 at 6:16 PM Rose Nguyen <rt...@google.com> wrote:
>>
>>
>> [image: Beam.png]
>>
>> February-March 2019 | Newsletter
>>
>> What’s been done
>>
>> ------------------------------
>>
>> Apache Beam 2.10.0 released (by: many contributors)
>>
>> -
>>
>> Download the release here.
>> <https://beam.apache.org/get-started/downloads/>
>> -
>>
>> See the blog post
>> <https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html> for more
>> details.
>>
>>
>> Apache Beam awarded the 2019 Technology of the Year Award!
>>
>> -
>>
>> InfoWorld just awarded Beam the 2019 Technology of the Year Award.
>> -
>>
>> See this article
>> <https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html?nsdr=true>
>> for more details.
>>
>>
>> Kettle Beam 0.5 released with support for flink (by: Matt Casters)
>>
>> -
>>
>> Kettle now supports Apache Flink as well as Cloud Dataflow and Spark.
>> -
>>
>> See Matt’s Blog
>> <http://sandbox.kettle.be/wordpress/index.php/2019/02/24/kettle-beam-update-0-5-0/>
>> for more details.
>>
>>
>>
>> What we’re working on...
>>
>> ------------------------------
>>
>> Apache Beam 2.11.0 release (by: many contributors)
>>
>>
>> Hive Metastore Table provider for SQL (by: Anton Kedin)
>>
>> -
>>
>> Support for plugging table providers through Beam SQL API to allow
>> obtaining table schemas from external sources.
>> -
>>
>> See the PR <https://github.com/apache/beam/pull/7746> for more
>> details.
>>
>>
>> User Defined Coders for the Beam Go SDK (by: Robert Burke)
>>
>> -
>>
>> Working on expanding the variety of user defined types that can be a
>> member of a PCollection in the Go SDK.
>> -
>>
>> See BEAM-3306 <https://issues.apache.org/jira/browse/BEAM-3306> for
>> more details.
>>
>>
>> Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen, Mark Liu,
>> Robbe Sneyders, Juta Staes, Valentyn Tymofieiev)
>>
>> -
>>
>> Beam 2.11.0 is the first release offering partial Python 3 support.
>> -
>>
>> Many thanks to all contributors who helped to reach this milestone.
>> -
>>
>> IO availablility on Python 3 is currently limited and only Python 3.5
>> version has been tested extensively.
>> -
>>
>> Stay tuned on BEAM-1251 for more details.
>>
>>
>> Notebooks for quickstarts and custom I/O (by: David Cavazos)
>>
>> -
>>
>> Adding IPython notebooks and snippets
>> -
>>
>> See [BEAM-6557] <https://github.com/apache/beam/pull/7679> for more
>> details.
>>
>>
>>
>>
>> New members
>> ------------------------------
>>
>> New PMC member!
>>
>> -
>>
>> Etienne Chauchot, Nantes, France
>>
>>
>> New Committers!
>>
>> -
>>
>> Gleb Kanterov, Stockholm, Sweden
>> -
>>
>> Michael Luckey
>>
>>
>> New Contributors!
>>
>> -
>>
>> Kyle Weaver, San Francisco, CA
>> -
>>
>> Would like to help begin implementing portability support for the
>> Spark runner
>> -
>>
>> Tanay Tummapalli, Delhi, India
>> -
>>
>> Would like to contribute to Open Source this summer as part of
>> Google Summer of Code
>> -
>>
>> Brian Hulette, Seattle, WA
>> -
>>
>> Contributing to Beam Portability
>> -
>>
>> Michał Walenia, Warsaw, Poland
>> -
>>
>> Working on integration and load testing
>> -
>>
>> Daniel Chen, San Francisco, CA
>> -
>>
>> Working on Beam Samza runner
>>
>>
>>
>> Talks & meetups
>> ------------------------------
>>
>>
>> Plugin Machine Intelligence and Apache Beam with Pentaho - Feb 7 @ London
>>
>> -
>>
>> Watch the How to Run Kettle on Apache Beam video here
>> <https://skillsmatter.com/skillscasts/13405-how-to-run-kettle-on-apache-beam#video>.
>>
>> -
>>
>> See event details here
>> <https://www.meetup.com/Pentaho-London-User-Group/events/256773962/>..
>>
>>
>> Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @ San Francisco,
>> CA
>>
>> -
>>
>> Organized by Thomas Weise and Austin Bennet, with speakers Tyler
>> Akidau, Robert Crowe, Thomas Weise and Amar Pai
>> -
>>
>> See event details here
>> <https://www.meetup.com/San-Francisco-Apache-Beam/events/257482350/>
>> and the slides for these presentation: Overview of Apache Beam and
>> TensorFlow Transform (TFX) with Apache Beam
>> <http://s.apache.org/beam-intro-feb-2019>, Python Streaming Pipelines
>> with Beam on Flink <http://go.lyft.com/python-flink-beam-meetup-2019>,
>> Dynamic pricing of Lyft rides using streaming
>> <https://www.slideshare.net/AmarPai2/dynamic-pricing-of-lyft-rides-using-streaming>
>>
>> .
>>
>> Flink meetup - Feb 21@ Seattle, WA
>>
>> -
>>
>> Speakers from Alibaba, Google, and Uber gave talks about Apache Flink
>> with Hive, Tensorflow, Beam, and AthenaX.
>> -
>>
>> See event details here
>> <https://www.meetup.com/seattle-flink/events/258723322/> and
>> presentations here <https://www.slideshare.net/BowenLi9/presentations>.
>>
>>
>>
>> Beam Summit Europe 2019 - June 19-20 @ Berlin
>>
>> -
>>
>> Beam Summit Europe 2019 will take place in Berlin on June 19-20.
>> -
>>
>> Speaker CfP and other details to follow soon!
>> -
>>
>> Twitter announcement!
>> <https://twitter.com/matthiasbaetens/status/1098854758893273088>
>>
>>
>>
>> Resources
>> ------------------------------
>>
>> Apache Jira Beginner’s Guide (by: Daniel Oliveira)
>>
>> -
>>
>> A guide
>> <https://cwiki.apache.org/confluence/display/BEAM/Beam+Jira+Beginner%27s+Guide>
>> to introduce Beam contributors to the basics of using the Apache Jira for
>> Beam development. Feedback welcomed!
>>
>>
>> An approach to community building from Apache Beam (by: Kenn Knowles)
>>
>> -
>>
>> The Apache Software Foundation has published committer guidelines to
>> help Beam's community building work.
>> -
>>
>> See the post <https://blogs.apache.org/comdev/date/20190222> on the
>> ASF blog.
>>
>>
>> Exploring Beam SQL on Google Cloud Platform (by: Graham Polley)
>>
>> -
>>
>> “In this article, I’ll dive into this new feature of Beam, and see
>> how it works by using a pipeline to read a data file from GCS, transform
>> it, and then perform a basic calculation on the values contained in the
>> file”.
>> -
>>
>> See article
>> <https://medium.com/weareservian/exploring-beam-sql-on-google-cloud-platform-b6c77f9b4af4>
>> and full source code
>> <https://github.com/polleyg/gcp-batch-ingestion-bigquery/blob/beam_sql/src/main/java/org/polleyg/BeamSQLMagic.java>
>> .
>>
>>
>> *Until Next Time!*
>> --
>> Rose Thị Nguyễn
>>
>>
Re: Apache Beam Newsletter - February/March 2019
Posted by Ismaël Mejía <ie...@gmail.com>.
We should have these newsletters published somewhere with a fixed URL so we
can add missing updates, I have at least missed putting stuff in the last 3
ones just to realize when the newsletter has been already 'published' (and
sadly interesting features like zstd have not been announced because of
this).
On Wed, Mar 6, 2019 at 2:48 PM Etienne Chauchot <ec...@apache.org>
wrote:
> Hi,
> I would add in what's been done:
>
> Work on cassandraIO (Etienne Chauchot, Mathieu Blanchard, Frank Shahar) :
> refactorings, bugfixes, new where clause, security fix
>
> Etienne
>
> Le lundi 04 mars 2019 à 18:36 +0100, Suneel Marthi a écrit :
>
> Is this the final draft? - we had 2 beam talks at Big Data Tech Warsaw
> last Wednesday - I can send the updates offline.
>
> On Mon, Mar 4, 2019 at 6:16 PM Rose Nguyen <rt...@google.com> wrote:
>
>
> [image: Beam.png]
>
> February-March 2019 | Newsletter
>
> What’s been done
>
> ------------------------------
>
> Apache Beam 2.10.0 released (by: many contributors)
>
> -
>
> Download the release here.
> <https://beam.apache.org/get-started/downloads/>
> -
>
> See the blog post
> <https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html> for more
> details.
>
>
> Apache Beam awarded the 2019 Technology of the Year Award!
>
> -
>
> InfoWorld just awarded Beam the 2019 Technology of the Year Award.
> -
>
> See this article
> <https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html?nsdr=true>
> for more details.
>
>
> Kettle Beam 0.5 released with support for flink (by: Matt Casters)
>
> -
>
> Kettle now supports Apache Flink as well as Cloud Dataflow and Spark.
> -
>
> See Matt’s Blog
> <http://sandbox.kettle.be/wordpress/index.php/2019/02/24/kettle-beam-update-0-5-0/>
> for more details.
>
>
>
> What we’re working on...
>
> ------------------------------
>
> Apache Beam 2.11.0 release (by: many contributors)
>
>
> Hive Metastore Table provider for SQL (by: Anton Kedin)
>
> -
>
> Support for plugging table providers through Beam SQL API to allow
> obtaining table schemas from external sources.
> -
>
> See the PR <https://github.com/apache/beam/pull/7746> for more details.
>
>
> User Defined Coders for the Beam Go SDK (by: Robert Burke)
>
> -
>
> Working on expanding the variety of user defined types that can be a
> member of a PCollection in the Go SDK.
> -
>
> See BEAM-3306 <https://issues.apache.org/jira/browse/BEAM-3306> for
> more details.
>
>
> Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen, Mark Liu, Robbe
> Sneyders, Juta Staes, Valentyn Tymofieiev)
>
> -
>
> Beam 2.11.0 is the first release offering partial Python 3 support.
> -
>
> Many thanks to all contributors who helped to reach this milestone.
> -
>
> IO availablility on Python 3 is currently limited and only Python 3.5
> version has been tested extensively.
> -
>
> Stay tuned on BEAM-1251 for more details.
>
>
> Notebooks for quickstarts and custom I/O (by: David Cavazos)
>
> -
>
> Adding IPython notebooks and snippets
> -
>
> See [BEAM-6557] <https://github.com/apache/beam/pull/7679> for more
> details.
>
>
>
>
> New members
> ------------------------------
>
> New PMC member!
>
> -
>
> Etienne Chauchot, Nantes, France
>
>
> New Committers!
>
> -
>
> Gleb Kanterov, Stockholm, Sweden
> -
>
> Michael Luckey
>
>
> New Contributors!
>
> -
>
> Kyle Weaver, San Francisco, CA
> -
>
> Would like to help begin implementing portability support for the
> Spark runner
> -
>
> Tanay Tummapalli, Delhi, India
> -
>
> Would like to contribute to Open Source this summer as part of
> Google Summer of Code
> -
>
> Brian Hulette, Seattle, WA
> -
>
> Contributing to Beam Portability
> -
>
> Michał Walenia, Warsaw, Poland
> -
>
> Working on integration and load testing
> -
>
> Daniel Chen, San Francisco, CA
> -
>
> Working on Beam Samza runner
>
>
>
> Talks & meetups
> ------------------------------
>
>
> Plugin Machine Intelligence and Apache Beam with Pentaho - Feb 7 @ London
>
> -
>
> Watch the How to Run Kettle on Apache Beam video here
> <https://skillsmatter.com/skillscasts/13405-how-to-run-kettle-on-apache-beam#video>.
>
> -
>
> See event details here
> <https://www.meetup.com/Pentaho-London-User-Group/events/256773962/>..
>
>
> Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @ San Francisco,
> CA
>
> -
>
> Organized by Thomas Weise and Austin Bennet, with speakers Tyler
> Akidau, Robert Crowe, Thomas Weise and Amar Pai
> -
>
> See event details here
> <https://www.meetup.com/San-Francisco-Apache-Beam/events/257482350/>
> and the slides for these presentation: Overview of Apache Beam and
> TensorFlow Transform (TFX) with Apache Beam
> <http://s.apache.org/beam-intro-feb-2019>, Python Streaming Pipelines
> with Beam on Flink <http://go.lyft.com/python-flink-beam-meetup-2019>, Dynamic
> pricing of Lyft rides using streaming
> <https://www.slideshare.net/AmarPai2/dynamic-pricing-of-lyft-rides-using-streaming>
>
> .
>
> Flink meetup - Feb 21@ Seattle, WA
>
> -
>
> Speakers from Alibaba, Google, and Uber gave talks about Apache Flink
> with Hive, Tensorflow, Beam, and AthenaX.
> -
>
> See event details here
> <https://www.meetup.com/seattle-flink/events/258723322/> and
> presentations here <https://www.slideshare.net/BowenLi9/presentations>.
>
>
>
> Beam Summit Europe 2019 - June 19-20 @ Berlin
>
> -
>
> Beam Summit Europe 2019 will take place in Berlin on June 19-20.
> -
>
> Speaker CfP and other details to follow soon!
> -
>
> Twitter announcement!
> <https://twitter.com/matthiasbaetens/status/1098854758893273088>
>
>
>
> Resources
> ------------------------------
>
> Apache Jira Beginner’s Guide (by: Daniel Oliveira)
>
> -
>
> A guide
> <https://cwiki.apache.org/confluence/display/BEAM/Beam+Jira+Beginner%27s+Guide>
> to introduce Beam contributors to the basics of using the Apache Jira for
> Beam development. Feedback welcomed!
>
>
> An approach to community building from Apache Beam (by: Kenn Knowles)
>
> -
>
> The Apache Software Foundation has published committer guidelines to
> help Beam's community building work.
> -
>
> See the post <https://blogs.apache.org/comdev/date/20190222> on the
> ASF blog.
>
>
> Exploring Beam SQL on Google Cloud Platform (by: Graham Polley)
>
> -
>
> “In this article, I’ll dive into this new feature of Beam, and see how
> it works by using a pipeline to read a data file from GCS, transform it,
> and then perform a basic calculation on the values contained in the file”.
> -
>
> See article
> <https://medium.com/weareservian/exploring-beam-sql-on-google-cloud-platform-b6c77f9b4af4>
> and full source code
> <https://github.com/polleyg/gcp-batch-ingestion-bigquery/blob/beam_sql/src/main/java/org/polleyg/BeamSQLMagic.java>
> .
>
>
> *Until Next Time!*
> --
> Rose Thị Nguyễn
>
>
Re: Apache Beam Newsletter - February/March 2019
Posted by Etienne Chauchot <ec...@apache.org>.
Hi,I would add in what's been done:
Work on cassandraIO (Etienne Chauchot, Mathieu Blanchard, Frank Shahar) : refactorings, bugfixes, new where clause,
security fix
Etienne
Le lundi 04 mars 2019 à 18:36 +0100, Suneel Marthi a écrit :
> Is this the final draft? - we had 2 beam talks at Big Data Tech Warsaw last Wednesday - I can send the updates
> offline.
> On Mon, Mar 4, 2019 at 6:16 PM Rose Nguyen <rt...@google.com> wrote:
> > February-March 2019 | Newsletter
> >
> > What’s been done
> >
> >
> > Apache Beam 2.10.0 released (by: many contributors)
> > Download the release here.See the blog post for more details.
> > Apache Beam awarded the 2019 Technology of the Year Award!
> > InfoWorld just awarded Beam the 2019 Technology of the Year Award.See this article for more details.
> > Kettle Beam 0.5 released with support for flink (by: Matt Casters)
> > Kettle now supports Apache Flink as well as Cloud Dataflow and Spark.See Matt’s Blog for more details.
> >
> >
> > What we’re working on...
> >
> >
> > Apache Beam 2.11.0 release (by: many contributors)
> >
> >
> > Hive Metastore Table provider for SQL (by: Anton Kedin)
> > Support for plugging table providers through Beam SQL API to allow obtaining table schemas from external sources.See
> > the PR for more details.
> > User Defined Coders for the Beam Go SDK (by: Robert Burke)
> > Working on expanding the variety of user defined types that can be a member of a PCollection in the Go SDK.See BEAM-
> > 3306 for more details.
> > Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen, Mark Liu, Robbe Sneyders, Juta Staes, Valentyn Tymofieiev)
> > Beam 2.11.0 is the first release offering partial Python 3 support. Many thanks to all contributors who helped to
> > reach this milestone.IO availablility on Python 3 is currently limited and only Python 3.5 version has been tested
> > extensively. Stay tuned on BEAM-1251 for more details.
> > Notebooks for quickstarts and custom I/O (by: David Cavazos)
> > Adding IPython notebooks and snippetsSee [BEAM-6557] for more details.
> >
> >
> > New members
> >
> >
> > New PMC member!
> > Etienne Chauchot, Nantes, France
> > New Committers!
> > Gleb Kanterov, Stockholm, SwedenMichael Luckey
> > New Contributors!
> > Kyle Weaver, San Francisco, CAWould like to help begin implementing portability support for the Spark runnerTanay
> > Tummapalli, Delhi, IndiaWould like to contribute to Open Source this summer as part of Google Summer of CodeBrian
> > Hulette, Seattle, WAContributing to Beam PortabilityMichał Walenia, Warsaw, PolandWorking on integration and load
> > testing Daniel Chen, San Francisco, CAWorking on Beam Samza runner
> >
> > Talks & meetups
> >
> >
> >
> > Plugin Machine Intelligence and Apache Beam with Pentaho - Feb 7 @ London
> > Watch the How to Run Kettle on Apache Beam video here. See event details here..
> > Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @ San Francisco, CA
> > Organized by Thomas Weise and Austin Bennet, with speakers Tyler Akidau, Robert Crowe, Thomas Weise and Amar PaiSee
> > event details here and the slides for these presentation: Overview of Apache Beam and TensorFlow Transform (TFX)
> > with Apache Beam, Python Streaming Pipelines with Beam on Flink, Dynamic pricing of Lyft rides using streaming.
> > Flink meetup - Feb 21@ Seattle, WA
> > Speakers from Alibaba, Google, and Uber gave talks about Apache Flink with Hive, Tensorflow, Beam, and AthenaX.See
> > event details here and presentations here.
> > Beam Summit Europe 2019 - June 19-20 @ Berlin
> > Beam Summit Europe 2019 will take place in Berlin on June 19-20.Speaker CfP and other details to follow soon!Twitter
> > announcement!
> >
> > Resources
> >
> >
> > Apache Jira Beginner’s Guide (by: Daniel Oliveira)
> > A guide to introduce Beam contributors to the basics of using the Apache Jira for Beam development. Feedback
> > welcomed!
> > An approach to community building from Apache Beam (by: Kenn Knowles)
> > The Apache Software Foundation has published committer guidelines to help Beam's community building work.See the
> > post on the ASF blog.
> > Exploring Beam SQL on Google Cloud Platform (by: Graham Polley)
> > “In this article, I’ll dive into this new feature of Beam, and see how it works by using a pipeline to read a data
> > file from GCS, transform it, and then perform a basic calculation on the values contained in the file”.See article
> > and full source code.
> >
> >
> > Until Next Time!--
> > Rose Thị Nguyễn
Re: Apache Beam Newsletter - February/March 2019
Posted by Suneel Marthi <sm...@apache.org>.
Is this the final draft? - we had 2 beam talks at Big Data Tech Warsaw last
Wednesday - I can send the updates offline.
On Mon, Mar 4, 2019 at 6:16 PM Rose Nguyen <rt...@google.com> wrote:
>
> [image: Beam.png]
>
> February-March 2019 | Newsletter
>
> What’s been done
>
> ------------------------------
>
> Apache Beam 2.10.0 released (by: many contributors)
>
> -
>
> Download the release here.
> <https://beam.apache.org/get-started/downloads/>
> -
>
> See the blog post
> <https://beam.apache.org/blog/2019/02/15/beam-2.10.0.html> for more
> details.
>
>
> Apache Beam awarded the 2019 Technology of the Year Award!
>
> -
>
> InfoWorld just awarded Beam the 2019 Technology of the Year Award.
> -
>
> See this article
> <https://www.infoworld.com/article/3336072/application-development/infoworlds-2019-technology-of-the-year-award-winners.html?nsdr=true>
> for more details.
>
>
> Kettle Beam 0.5 released with support for flink (by: Matt Casters)
>
> -
>
> Kettle now supports Apache Flink as well as Cloud Dataflow and Spark.
> -
>
> See Matt’s Blog
> <http://sandbox.kettle.be/wordpress/index.php/2019/02/24/kettle-beam-update-0-5-0/>
> for more details.
>
>
>
> What we’re working on...
>
> ------------------------------
>
> Apache Beam 2.11.0 release (by: many contributors)
>
>
> Hive Metastore Table provider for SQL (by: Anton Kedin)
>
> -
>
> Support for plugging table providers through Beam SQL API to allow
> obtaining table schemas from external sources.
> -
>
> See the PR <https://github.com/apache/beam/pull/7746> for more details.
>
>
> User Defined Coders for the Beam Go SDK (by: Robert Burke)
>
> -
>
> Working on expanding the variety of user defined types that can be a
> member of a PCollection in the Go SDK.
> -
>
> See BEAM-3306 <https://issues.apache.org/jira/browse/BEAM-3306> for
> more details.
>
>
> Python 3 (by: Ahmet Altay, Robert Bradshaw, Charles Chen, Mark Liu, Robbe
> Sneyders, Juta Staes, Valentyn Tymofieiev)
>
> -
>
> Beam 2.11.0 is the first release offering partial Python 3 support.
> -
>
> Many thanks to all contributors who helped to reach this milestone.
> -
>
> IO availablility on Python 3 is currently limited and only Python 3.5
> version has been tested extensively.
> -
>
> Stay tuned on BEAM-1251 for more details.
>
>
> Notebooks for quickstarts and custom I/O (by: David Cavazos)
>
> -
>
> Adding IPython notebooks and snippets
> -
>
> See [BEAM-6557] <https://github.com/apache/beam/pull/7679> for more
> details.
>
>
>
>
> New members
> ------------------------------
>
> New PMC member!
>
> -
>
> Etienne Chauchot, Nantes, France
>
>
> New Committers!
>
> -
>
> Gleb Kanterov, Stockholm, Sweden
> -
>
> Michael Luckey
>
>
> New Contributors!
>
> -
>
> Kyle Weaver, San Francisco, CA
> -
>
> Would like to help begin implementing portability support for the
> Spark runner
> -
>
> Tanay Tummapalli, Delhi, India
> -
>
> Would like to contribute to Open Source this summer as part of
> Google Summer of Code
> -
>
> Brian Hulette, Seattle, WA
> -
>
> Contributing to Beam Portability
> -
>
> Michał Walenia, Warsaw, Poland
> -
>
> Working on integration and load testing
> -
>
> Daniel Chen, San Francisco, CA
> -
>
> Working on Beam Samza runner
>
>
>
> Talks & meetups
> ------------------------------
>
>
> Plugin Machine Intelligence and Apache Beam with Pentaho - Feb 7 @ London
>
> -
>
> Watch the How to Run Kettle on Apache Beam video here
> <https://skillsmatter.com/skillscasts/13405-how-to-run-kettle-on-apache-beam#video>.
>
> -
>
> See event details here
> <https://www.meetup.com/Pentaho-London-User-Group/events/256773962/>..
>
>
> Beam @Lyft / Streaming, TensorFlow and use-cases - Feb 7 @ San Francisco,
> CA
>
> -
>
> Organized by Thomas Weise and Austin Bennet, with speakers Tyler
> Akidau, Robert Crowe, Thomas Weise and Amar Pai
> -
>
> See event details here
> <https://www.meetup.com/San-Francisco-Apache-Beam/events/257482350/>
> and the slides for these presentation: Overview of Apache Beam and
> TensorFlow Transform (TFX) with Apache Beam
> <http://s.apache.org/beam-intro-feb-2019>, Python Streaming Pipelines
> with Beam on Flink <http://go.lyft.com/python-flink-beam-meetup-2019>, Dynamic
> pricing of Lyft rides using streaming
> <https://www.slideshare.net/AmarPai2/dynamic-pricing-of-lyft-rides-using-streaming>
>
> .
>
> Flink meetup - Feb 21@ Seattle, WA
>
> -
>
> Speakers from Alibaba, Google, and Uber gave talks about Apache Flink
> with Hive, Tensorflow, Beam, and AthenaX.
> -
>
> See event details here
> <https://www.meetup.com/seattle-flink/events/258723322/> and
> presentations here <https://www.slideshare.net/BowenLi9/presentations>.
>
>
>
> Beam Summit Europe 2019 - June 19-20 @ Berlin
>
> -
>
> Beam Summit Europe 2019 will take place in Berlin on June 19-20.
> -
>
> Speaker CfP and other details to follow soon!
> -
>
> Twitter announcement!
> <https://twitter.com/matthiasbaetens/status/1098854758893273088>
>
>
>
> Resources
> ------------------------------
>
> Apache Jira Beginner’s Guide (by: Daniel Oliveira)
>
> -
>
> A guide
> <https://cwiki.apache.org/confluence/display/BEAM/Beam+Jira+Beginner%27s+Guide>
> to introduce Beam contributors to the basics of using the Apache Jira for
> Beam development. Feedback welcomed!
>
>
> An approach to community building from Apache Beam (by: Kenn Knowles)
>
> -
>
> The Apache Software Foundation has published committer guidelines to
> help Beam's community building work.
> -
>
> See the post <https://blogs.apache.org/comdev/date/20190222> on the
> ASF blog.
>
>
> Exploring Beam SQL on Google Cloud Platform (by: Graham Polley)
>
> -
>
> “In this article, I’ll dive into this new feature of Beam, and see how
> it works by using a pipeline to read a data file from GCS, transform it,
> and then perform a basic calculation on the values contained in the file”.
> -
>
> See article
> <https://medium.com/weareservian/exploring-beam-sql-on-google-cloud-platform-b6c77f9b4af4>
> and full source code
> <https://github.com/polleyg/gcp-batch-ingestion-bigquery/blob/beam_sql/src/main/java/org/polleyg/BeamSQLMagic.java>
> .
>
>
> *Until Next Time!*
> --
> Rose Thị Nguyễn
>