You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/04/01 20:39:25 UTC
[jira] [Commented] (FLINK-3428) Add fixed time trailing
timestamp/watermark extractor
[ https://issues.apache.org/jira/browse/FLINK-3428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15222140#comment-15222140 ]
ASF GitHub Bot commented on FLINK-3428:
---------------------------------------
Github user StephanEwen commented on the pull request:
https://github.com/apache/flink/pull/1764#issuecomment-204508742
The code looks good now.
- How about we move all the convenience timestamp extractors / watermark generators into a separate packe `.functions.timestamps`?
**Docs**
I am a bit unsure about the docs. They seem to get confusing now. What is a mechanism (periodic / punctuated), what is just a convenience implementation (Ascending, BoundedOutOfOrderness).
How about leaving the docs page on watermark generation as it is and add another sub-page that lists pre-defined watermark generators. We would list all the future watermark generators there:
- ascending timestamps
- bounded out-of-orderness
- historgam based
> Add fixed time trailing timestamp/watermark extractor
> -----------------------------------------------------
>
> Key: FLINK-3428
> URL: https://issues.apache.org/jira/browse/FLINK-3428
> Project: Flink
> Issue Type: Improvement
> Reporter: Robert Metzger
> Assignee: Kostas Kloudas
>
> Flink currently provides only one build-in timestamp extractor, which assumes strictly ascending timestamps. In real world use cases, timestamps are almost never strictly ascending.
> Therefore, I propose to provide an utility watermark extractor which is generating watermarks with a fixed-time trailing.
> The implementation should keep track of the highest event-time seen so far and subtract a fixed amount of time from that event time.
> This way, users can for example specify that the watermarks should always "lag behind" 10 minutes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)