You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@storm.apache.org by "冯健 (JIRA)" <ji...@apache.org> on 2015/11/18 04:09:12 UTC

[jira] [Commented] (STORM-1187) Support for late and out of order events in time based windows

    [ https://issues.apache.org/jira/browse/STORM-1187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15010117#comment-15010117 ] 

冯健 commented on STORM-1187:
---------------------------

In this paper (http://www.cs.cmu.edu/~pavlo/courses/fall2013/static/papers/p734-akidau.pdf) from google which describes a framework for building realtime computation application,they provide a semantic named "Low Watermarks" which indicates that all data up to a given timestamp(generation time) has been received. Using this semantic,we can distinguish whether the events are simply delayed or actually not there, also, out-of-order events won't be a problem anymore.

> Support for late and out of order events in time based windows
> --------------------------------------------------------------
>
>                 Key: STORM-1187
>                 URL: https://issues.apache.org/jira/browse/STORM-1187
>             Project: Apache Storm
>          Issue Type: Sub-task
>            Reporter: Arun Mahadevan
>            Assignee: Arun Mahadevan
>
> Right now the time based windows uses the timestamp when the tuple is received by the bolt. 
> However there are use cases where the tuples can be processed based on the time when they are actually generated vs the time when they are received. So we need to add support for processing events with a time lag and also have some way to specify and read tuple timestamps.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)