Breaking changes in the Streaming API

[Stephan Ewen <se...@apache.org>]

Hi everyone!

There are two remaining issues right now pending for 1.0 that will cause
breaking API changes in the Streaming API.


1)
[FLINK-3379]

The Timestamp Extractor needs to be changed. That really seems necessary,
based on the user feedback, because a lot of people mentioned that they are
getting confused about the TimestampExtractor's mixed two-way system of
generating watermarks.

The issue suggests to pull the two different modes of generating watermarks
into two different classes.


2)

[FLINK-3371] makes the "Trigger" an abstract class (currently interface)
and moves the "TriggerResult" to a dedicated class. This is necessary for
avoiding breaking changes in the future, after the release.

The reason why for these changes are "aligned windows", which have one
Trigger for the entire window across all keys (
https://issues.apache.org/jira/browse/FLINK-3370)

Aligned windows are for example most sliding/tumbling time windows, while
unaligned windows (with a trigger per key) are for example session and
count windows. For aligned windows, we can implement an optimized
representation that uses less memory and is more lightweight to checkpoint.

Also, the Trigger class may evolve a bit, and and with an abstract class we
can add methods without breaking user-defined Triggers in the future.


Greetings,
Stephan

Re: Breaking changes in the Streaming API

[Robert Metzger <rm...@apache.org>]

big +1 on reworking the timestamp extractor. There were too many users
stumbling across this. I also misunderstood it when trying it out the first
time.

On Tue, Feb 9, 2016 at 4:54 PM, Stephan Ewen <se...@apache.org> wrote:

> Hi everyone!
>
> There are two remaining issues right now pending for 1.0 that will cause
> breaking API changes in the Streaming API.
>
>
> 1)
> [FLINK-3379]
>
> The Timestamp Extractor needs to be changed. That really seems necessary,
> based on the user feedback, because a lot of people mentioned that they are
> getting confused about the TimestampExtractor's mixed two-way system of
> generating watermarks.
>
> The issue suggests to pull the two different modes of generating watermarks
> into two different classes.
>
>
> 2)
>
> [FLINK-3371] makes the "Trigger" an abstract class (currently interface)
> and moves the "TriggerResult" to a dedicated class. This is necessary for
> avoiding breaking changes in the future, after the release.
>
> The reason why for these changes are "aligned windows", which have one
> Trigger for the entire window across all keys (
> https://issues.apache.org/jira/browse/FLINK-3370)
>
> Aligned windows are for example most sliding/tumbling time windows, while
> unaligned windows (with a trigger per key) are for example session and
> count windows. For aligned windows, we can implement an optimized
> representation that uses less memory and is more lightweight to checkpoint.
>
> Also, the Trigger class may evolve a bit, and and with an abstract class we
> can add methods without breaking user-defined Triggers in the future.
>
>
> Greetings,
> Stephan
>