You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@beam.apache.org by Shen Li <cs...@gmail.com> on 2016/07/06 15:01:22 UTC

Window Strategy for KeyedPCollectionTuples and CoGroupByKey

Hi,

If the PCollections in a KeyedPCollectionTuple have different window
strategies (WindowFn, Trigger, etc.), how does a CoGroupByKey work? When
will it be triggered? How does it determine which kvs from each PCollection
to co-group?

Thanks,

Shen

Re: Window Strategy for KeyedPCollectionTuples and CoGroupByKey

Posted by Shen Li <cs...@gmail.com>.

Hi Robert,

Thanks for your response.

Regards,

Shen

On Wed, Jul 6, 2016 at 4:02 PM, Robert Bradshaw <robertwb@google.com.invalid
> wrote:

> It is an error at pipeline construction time to use CoGroupByKey with
> differing windowing strategies. If you want to do such joins, you may
> want to look into using side inputs which are more flexible.
>
> On Wed, Jul 6, 2016 at 8:01 AM, Shen Li <cs...@gmail.com> wrote:
> > Hi,
> >
> > If the PCollections in a KeyedPCollectionTuple have different window
> > strategies (WindowFn, Trigger, etc.), how does a CoGroupByKey work? When
> > will it be triggered? How does it determine which kvs from each
> PCollection
> > to co-group?
> >
> > Thanks,
> >
> > Shen
>

Re: Window Strategy for KeyedPCollectionTuples and CoGroupByKey

Posted by Robert Bradshaw <ro...@google.com.INVALID>.

It is an error at pipeline construction time to use CoGroupByKey with
differing windowing strategies. If you want to do such joins, you may
want to look into using side inputs which are more flexible.

On Wed, Jul 6, 2016 at 8:01 AM, Shen Li <cs...@gmail.com> wrote:
> Hi,
>
> If the PCollections in a KeyedPCollectionTuple have different window
> strategies (WindowFn, Trigger, etc.), how does a CoGroupByKey work? When
> will it be triggered? How does it determine which kvs from each PCollection
> to co-group?
>
> Thanks,
>
> Shen