[jira] [Updated] (KUDU-2673) Event timestamp support with kudu.

["Xu Yao (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KUDU-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Xu Yao updated KUDU-2673:
-------------------------
    Labels: features roadmap-candidate  (was: features)

> Event timestamp support with kudu.
> ----------------------------------
>
>                 Key: KUDU-2673
>                 URL: https://issues.apache.org/jira/browse/KUDU-2673
>             Project: Kudu
>          Issue Type: Improvement
>          Components: java, spark, tserver
>    Affects Versions: 1.8.0
>            Reporter: yangz
>            Priority: Major
>              Labels: features, roadmap-candidate
>             Fix For: 1.8.0
>
>
> Kudu has the ability to read historical data. But it is based by the timestamp produced by kudu transaction and mvcc system. The timestamp kudu used greatly weakened the usability.
> For our use case. we write data to kudu from data stream. We use range partition by day.
> We want to get the hour version from kudu. So we need read history data from kudu.
> It produced by undo file. But when user give a timestamp, it means timestamp the event happen, associated with the data. Not the timestamp kudu produced. So we need a way to set event timestamp to the kudu system.
> Finally, we got a way to solve this problem.
> But our solution has two limit.
>  # We only update the table by a row, and for one row we have a timestamp with it.
>  # For getting the right history version of data, we need the data stream send data by event time order.
> Despite these problems, it has satisfied our current business.
>  
> And our implement also solve part problem for the wrong order problem of event time if you only need the newest data, which will not read undo file.
> for the data send into kudu,       t1 < t2
> t1 upsert -> t2 upsert      ->    newest will be t2 value
> t2 upsert -> t1 upsret      ->    (current kudu implement) t1,  our implement will be t2.
>  
> Maybe our solution is not the best for the problem. But I think kudu snapshot read should support event time.
> Our solution is not so complete for all user cases. But I hope it will be useful for some cases with the community.   
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)