You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Shen Li <cs...@gmail.com> on 2016/07/25 15:55:38 UTC

Help understand how Flink Runner translate triggering information

Hi,

I am trying to understand how Flink Runner tells the Flink system about the
triggers defined using Beam API. In the source code of Flink runner, the
WindowBoundTranslator passes the windowingStrategy to the
FlinkParDoBoundWrapper which does not seem to use it? How is the triggering
information passed to Flink?

Thanks,

Shen

Re: Help understand how Flink Runner translate triggering information

Posted by Shen Li <cs...@gmail.com>.
Hi Aljoscha,

Thanks a lot for your help!

Shen

On Mon, Jul 25, 2016 at 12:31 PM, Aljoscha Krettek <al...@apache.org>
wrote:

> Hi,
> for that you would have to look at how Combine.PerKey and GroupByKey are
> translated. We use a GroupAlsoByWindowViaWindowSetDoFn that internally uses
> a ReduceFnRunner to manage all the windowing. The windowing strategy as
> well as the SystemReduceFn is passed to
> GroupAlsoByWindowViaWindowSetDoFn.create() to create an actual instance of
>  GroupAlsoByWindowViaWindowSetDoFn.
>
> Cheers,
> Aljoscha
>
> On Mon, 25 Jul 2016 at 17:55 Shen Li <cs...@gmail.com> wrote:
>
> > Hi,
> >
> > I am trying to understand how Flink Runner tells the Flink system about
> the
> > triggers defined using Beam API. In the source code of Flink runner, the
> > WindowBoundTranslator passes the windowingStrategy to the
> > FlinkParDoBoundWrapper which does not seem to use it? How is the
> triggering
> > information passed to Flink?
> >
> > Thanks,
> >
> > Shen
> >
>

Re: Help understand how Flink Runner translate triggering information

Posted by Aljoscha Krettek <al...@apache.org>.
Hi,
for that you would have to look at how Combine.PerKey and GroupByKey are
translated. We use a GroupAlsoByWindowViaWindowSetDoFn that internally uses
a ReduceFnRunner to manage all the windowing. The windowing strategy as
well as the SystemReduceFn is passed to
GroupAlsoByWindowViaWindowSetDoFn.create() to create an actual instance of
 GroupAlsoByWindowViaWindowSetDoFn.

Cheers,
Aljoscha

On Mon, 25 Jul 2016 at 17:55 Shen Li <cs...@gmail.com> wrote:

> Hi,
>
> I am trying to understand how Flink Runner tells the Flink system about the
> triggers defined using Beam API. In the source code of Flink runner, the
> WindowBoundTranslator passes the windowingStrategy to the
> FlinkParDoBoundWrapper which does not seem to use it? How is the triggering
> information passed to Flink?
>
> Thanks,
>
> Shen
>