You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@kylin.apache.org by "liyang (JIRA)" <ji...@apache.org> on 2016/05/28 01:07:12 UTC

[jira] [Created] (KYLIN-1744) Separate concepts of source offset and date range on cube segments

liyang created KYLIN-1744:
-----------------------------

             Summary: Separate concepts of source offset and date range on cube segments
                 Key: KYLIN-1744
                 URL: https://issues.apache.org/jira/browse/KYLIN-1744
             Project: Kylin
          Issue Type: Sub-task
            Reporter: liyang


Previously date range is used to 1) cut data from source; 2) mark min/max data time for segment pruning. However for streaming source, these two concepts are separate. E.g. offset is used to cut data from Kafka; and the min/max data time of segments can overlap due to late coming records.

Will add two more attributes in CubeSegment, sourceOffsetStart and sourceOffsetEnd. To be backward compatible, when the two attributes are missing (equals to 0), dateRangeStart and dateRangeEnd will serve as source offsets.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)