[jira] [Commented] (FLINK-3379) Refactor TimestampExtractor

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-3379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149103#comment-15149103 ] 

ASF GitHub Bot commented on FLINK-3379:
---------------------------------------

GitHub user StephanEwen opened a pull request:

    https://github.com/apache/flink/pull/1646

    [FLINK-3379] [FLINK-3415] [streaming] Refactor TimestampExtractor

    This pull request divides the TimestampExtractor into two separate classes
     - one class handled periodic watermarks
     - the other class handled watermarks triggered by individual elements
    
    This also makes sure that any timestamp assigner / watermark generators cannot generate
    negative watermarks.
    
    This also adds tests for the operators that run the timestamp extractors.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/StephanEwen/incubator-flink refactor_extractors

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/1646.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1646
    
----
commit 1e3c4597dd1785430735950eef2b7b8c2154faaf
Author: Stephan Ewen <se...@apache.org>
Date:   2016-02-16T09:37:59Z

    [FLINK-3379] [FLINK-3415] [streaming] Refactor TimestampExtractor into two separate classes
    
     - one class handled periodic watermarks
     - the other class handled watermarks triggered by elements
    
    This also makes sure that any timestamp assigner / watermark generators cannot generate
    negative watermarks

----


> Refactor TimestampExtractor
> ---------------------------
>
>                 Key: FLINK-3379
>                 URL: https://issues.apache.org/jira/browse/FLINK-3379
>             Project: Flink
>          Issue Type: Improvement
>          Components: Streaming
>    Affects Versions: 0.10.1
>            Reporter: Stephan Ewen
>            Assignee: Stephan Ewen
>            Priority: Critical
>             Fix For: 1.0.0
>
>
> Based on a lot of user feedback, the current {{TimestampExtractor}} seems very confusing. It implements simultaneously two modes of generating watermarks:
>   - Each record that passes through can decide to cause a watermark.
>   - The timestamp extractor can define a certain watermark timestamp which is periodically picked up by the system and triggers a watermark (if larger than the previous watermark).
> Figuring out how these modes interplay, and how to define the methods to only use one mode has been quite an obstacle for several users. We should break this class into two different classes, one per mode of generating watermarks, to make it easier to understand.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)