You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Robert Metzger (JIRA)" <ji...@apache.org> on 2016/02/17 12:08:18 UTC

[jira] [Created] (FLINK-3429) Add histogram-based timestamp/watermark extractor

Robert Metzger created FLINK-3429:
-------------------------------------

             Summary: Add histogram-based timestamp/watermark extractor
                 Key: FLINK-3429
                 URL: https://issues.apache.org/jira/browse/FLINK-3429
             Project: Flink
          Issue Type: Improvement
          Components: Streaming
            Reporter: Robert Metzger


While FLINK-3428 provides a static low watermark generator, we should also provide a watermark extractor which periodically takes samples of the event-time distribution in a data stream into a histogram.
>From that histogram, users can specify that watermarks should cover a certain percentile of the data.

For example a user might want watermarks to rise when 90% of the events have been seen. With the periodic update of the histogram, the extractor can react to network latencies and other issues causing late, out-of-order arrivals.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)