You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Shen Li <cs...@gmail.com> on 2016/08/15 14:25:40 UTC

Questions about IdentityWindowFn

Hi,

I am a little confused about the usage of internal IdentityWindowFn. My
current understanding is that it should be used when a transform wants to
change window configurations (e.g., triggering, lateness, etc.) without
modifying the window assignments. Is that correct? If so, how come
IdentifyWindowFn.assignWindows() returns a collection of a single
BoundedWindow? What if the upstream WindowFn assigns an element to multiple
windows (e.g., SlidingWindows)?

Thanks,

Shen

Re: Questions about IdentityWindowFn

Posted by Shen Li <cs...@gmail.com>.
Hi Lukasz,

Thanks for the explanation.

Shen

On Mon, Aug 15, 2016 at 12:02 PM, Lukasz Cwik <lc...@google.com.invalid>
wrote:

> Some areas of the Beam model in the SDK alluded to the use of a compressed
> representation of an element along with the set of windows it is assigned
> to. However, the model itself views elements in different windows as fully
> independent, so the SDK should not place any obligation on the part of the
> runner or user to use a particular representation.
>
> This change
> <https://github.com/apache/incubator-beam/commit/
> 08104410177063b1095bd91b24b40f9961c92cf2>
> removed those places in the SDK where an element is treated in multiple
> windows at once. This was done by exploding the compressed representation
> so [(W1, W2), V] became [W1, V] and [W2, V]
>
> On Mon, Aug 15, 2016 at 7:25 AM, Shen Li <cs...@gmail.com> wrote:
>
> > Hi,
> >
> > I am a little confused about the usage of internal IdentityWindowFn. My
> > current understanding is that it should be used when a transform wants to
> > change window configurations (e.g., triggering, lateness, etc.) without
> > modifying the window assignments. Is that correct? If so, how come
> > IdentifyWindowFn.assignWindows() returns a collection of a single
> > BoundedWindow? What if the upstream WindowFn assigns an element to
> multiple
> > windows (e.g., SlidingWindows)?
> >
> > Thanks,
> >
> > Shen
> >
>

Re: Questions about IdentityWindowFn

Posted by Lukasz Cwik <lc...@google.com.INVALID>.
Some areas of the Beam model in the SDK alluded to the use of a compressed
representation of an element along with the set of windows it is assigned
to. However, the model itself views elements in different windows as fully
independent, so the SDK should not place any obligation on the part of the
runner or user to use a particular representation.

This change
<https://github.com/apache/incubator-beam/commit/08104410177063b1095bd91b24b40f9961c92cf2>
removed those places in the SDK where an element is treated in multiple
windows at once. This was done by exploding the compressed representation
so [(W1, W2), V] became [W1, V] and [W2, V]

On Mon, Aug 15, 2016 at 7:25 AM, Shen Li <cs...@gmail.com> wrote:

> Hi,
>
> I am a little confused about the usage of internal IdentityWindowFn. My
> current understanding is that it should be used when a transform wants to
> change window configurations (e.g., triggering, lateness, etc.) without
> modifying the window assignments. Is that correct? If so, how come
> IdentifyWindowFn.assignWindows() returns a collection of a single
> BoundedWindow? What if the upstream WindowFn assigns an element to multiple
> windows (e.g., SlidingWindows)?
>
> Thanks,
>
> Shen
>