You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@kylin.apache.org by "liyang (JIRA)" <ji...@apache.org> on 2016/05/29 14:11:12 UTC

[jira] [Resolved] (KYLIN-1744) Separate concepts of source offset and date range on cube segments

     [ https://issues.apache.org/jira/browse/KYLIN-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

liyang resolved KYLIN-1744.
---------------------------
       Resolution: Fixed
    Fix Version/s: v1.5.3

> Separate concepts of source offset and date range on cube segments
> ------------------------------------------------------------------
>
>                 Key: KYLIN-1744
>                 URL: https://issues.apache.org/jira/browse/KYLIN-1744
>             Project: Kylin
>          Issue Type: Sub-task
>            Reporter: liyang
>            Assignee: liyang
>             Fix For: v1.5.3
>
>
> Previously date range is used to 1) cut data from source; 2) mark min/max data time for segment pruning. However for streaming source, these two concepts are separate. E.g. offset is used to cut data from Kafka; and the min/max data time of segments can overlap due to late coming records.
> Will add two more attributes in CubeSegment, sourceOffsetStart and sourceOffsetEnd. To be backward compatible, when the two attributes are missing (equals to 0), dateRangeStart and dateRangeEnd will serve as source offsets.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)