[ANNOUNCE] A brand new DoFn @ HEAD (action required)

[Kenneth Knowles <kl...@google.com>]

Hi all,

We have just incorporated a major feature into the master branch of the
Beam codebase: a new DoFn. Assuming you are tracking HEAD, this will
require some (easy) adjustment to your pipelines.

The experimental feature that was in the codebase as DoFnWithContext is now
just "DoFn" for Beam. It is a radical enabler of new features, most
excitingly our design for stateful DoFn and timer-based callbacks. But does
require you to do some search-and-replace in your pipelines.

For a DoFn like this:

    new DoFn<Foo, Baz>() {
      @Override
      public void processElement(ProcessContext c) { … }
    }

now you'll need to write this:

    new DoFn<Foo, Baz>() {
      @ProcessElement   // <-- This is the only difference
      public void processElement(ProcessContext c) { … }
    }

Likewise for your startBundle() and finishBundle() methods, you'll replace
@Override with @StartBundle and @FinishBundle, respectively. If you want to
see more, we have ported a number of examples and tests to use this new
DoFn and will be proceeding with the rest and the documentation.

If you don't want to make any change right now, you may want to delay
tracking HEAD. I have tagged the last commit prior to the change so you can
`git checkout OldDoFn` and move beyond that point when you are ready.

Kenn

Re: [ANNOUNCE] A brand new DoFn @ HEAD (action required)

[Dan Halperin <dh...@google.com>]

Thanks Kenn! I am impressed that this was doable so quickly -- only a week
after the Beam dev community +1ed the heck out of it. This is going to be
great.

(More info on the original dev thread:
http://mail-archives.apache.org/mod_mbox/incubator-beam-dev/201607.mbox/%3CCAN_Ypr3P1xmVL9O7kvy6LSMeNz7k3Dpk9QaSt8UmV686ZzuKaQ%40mail.gmail.com%3E
)

On Wed, Aug 3, 2016 at 6:44 PM, Kenneth Knowles <kl...@google.com> wrote:

> Hi all,
>
> We have just incorporated a major feature into the master branch of the
> Beam codebase: a new DoFn. Assuming you are tracking HEAD, this will
> require some (easy) adjustment to your pipelines.
>
> The experimental feature that was in the codebase as DoFnWithContext is
> now just "DoFn" for Beam. It is a radical enabler of new features, most
> excitingly our design for stateful DoFn and timer-based callbacks. But does
> require you to do some search-and-replace in your pipelines.
>
> For a DoFn like this:
>
>     new DoFn<Foo, Baz>() {
>       @Override
>       public void processElement(ProcessContext c) { … }
>     }
>
> now you'll need to write this:
>
>     new DoFn<Foo, Baz>() {
>       @ProcessElement   // <-- This is the only difference
>       public void processElement(ProcessContext c) { … }
>     }
>
> Likewise for your startBundle() and finishBundle() methods, you'll replace
> @Override with @StartBundle and @FinishBundle, respectively. If you want to
> see more, we have ported a number of examples and tests to use this new
> DoFn and will be proceeding with the rest and the documentation.
>
> If you don't want to make any change right now, you may want to delay
> tracking HEAD. I have tagged the last commit prior to the change so you can
> `git checkout OldDoFn` and move beyond that point when you are ready.
>
> Kenn
>