You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/29 21:43:12 UTC

[GitHub] [beam] robertwb commented on a diff in pull request #17723: [BEAM-11103] Add blog post for go 2.40 release

robertwb commented on code in PR #17723:
URL: https://github.com/apache/beam/pull/17723#discussion_r910436897


##########
website/www/site/content/en/blog/go-2.40.md:
##########
@@ -0,0 +1,69 @@
+---
+title:  "Big Improvements in Beam Go's 2.40 Release"
+date:   2022-05-17 00:00:01 -0800
+categories:
+  - blog
+  - go
+authors:
+  - damccorm
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+The 2.40 release is one of Beam Go's biggest yet, and we wanted to highlight
+some of the biggest changes coming with this important release!
+
+# Native Streaming Support
+
+2.40 marks the release of one of our most anticipated feature sets yet:
+native streaming Go pipelines. This includes adding support for:
+
+- [Self Checkpointing](https://beam.apache.org/documentation/programming-guide/#user-initiated-checkpoint)
+- [Watermark Estimation](https://beam.apache.org/documentation/programming-guide/#watermark-estimation)
+- [Pipeline Drain/Truncation](https://beam.apache.org/documentation/programming-guide/#truncating-during-drain)
+- [Bundle Finalization](https://beam.apache.org/documentation/programming-guide/#bundle-finalization) (added in 2.39)
+
+With all of these features, it is now possible to write your own streaming
+pipeline source DoFns in Go without relying on cross-language transforms
+from Java or Python. We encourage you to try out all of these new features
+in your streaming pipelines! The [programming guide](https://beam.apache.org/documentation/programming-guide/#splittable-dofns)
+has additional information on getting started with native Go streaming DoFns.
+
+# Generic Registration (Make Your Pipelines 3x Faster)
+
+The release of [Go Generics](https://go.dev/blog/intro-generics) in Go 1.18
+unlocked significant performance improvements for Beam Go. With generics,
+we were able to add simple registration functions that can massively reduce
+your pipeline's runtime and resource consumption. For example, registering
+the ParDo's in our load tests which are designed to simulate a basic pipeline
+reduced execution time from around 25 minutes to around 7 minutes on average
+- an over 70% reduction!
+
+<img class="center-block"
+     src="/images/blog/go-registration.png"
+     alt="Beam Registration Load Tests ParDo Improvements">
+
+To get started with registering your own DoFns and unlocking these performance
+gains, check out the [registration doc page](https://pkg.go.dev/github.com/apache/beam/sdks/go/pkg/beam/register).
+
+# What's Next?
+
+Moving forward, we remain focused on improving the streaming experience and
+leveraging generics to improve the Sdk. Specific improvements we are considering
+include adding [State & Timers](https://beam.apache.org/documentation/programming-guide/#state-and-timers)
+support, introducing a Go expansion service so that Go DoFns can be used in other
+languages, wrapping more Java and Python IOs so that they can be easily used in Go,
+and exploring the use of generics to simplify the user experience. As always,

Review Comment:
   "exploring the use of generics to simplify the user experience" is somewhat redundant with "leveraging generics to improve the Sdk" above. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org