You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Victoria Bialas (Jira)" <ji...@apache.org> on 2019/10/25 17:58:00 UTC

[jira] [Comment Edited] (KAFKA-6453) Reconsider timestamp propagation semantics

    [ https://issues.apache.org/jira/browse/KAFKA-6453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16959956#comment-16959956 ] 

Victoria Bialas edited comment on KAFKA-6453 at 10/25/19 5:57 PM:
------------------------------------------------------------------

Helpful links:
 * Related KIP-258: [https://cwiki.apache.org/confluence/display/KAFKA/KIP-258%3A+Allow+to+Store+Record+Timestamps+in+RocksDB]
 * Earlier KIP (upon which this new work is based) KIP-251: [https://cwiki.apache.org/confluence/display/KAFKA/KIP-251%3A+Allow+timestamp+manipulation+in+Processor+API]
 * Docs in Confluent pertinent to WindowStore: [Windowing|#windowing]

 

 


was (Author: orangesnap):
Helpful links:
 * Related KIP: [KIP-258|[https://cwiki.apache.org/confluence/display/KAFKA/KIP-258%3A+Allow+to+Store+Record+Timestamps+in+RocksDB]]
 * Earlier KIP (upon which this new work is based):  [KIP-251]([https://cwiki.apache.org/confluence/display/KAFKA/KIP-251%3A+Allow+timestamp+manipulation+in+Processor+API])
 * Docs in Confluent pertinent to WindowStore: [Windowing|#windowing]]

 

 

> Reconsider timestamp propagation semantics
> ------------------------------------------
>
>                 Key: KAFKA-6453
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6453
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Matthias J. Sax
>            Assignee: Victoria Bialas
>            Priority: Major
>              Labels: needs-kip
>
> Atm, Kafka Streams only has a defined "contract" about timestamp propagation at the Processor API level: all processor within a sub-topology, see the timestamp from the input topic record and this timestamp will be used for all result record when writing them to an topic, too.
> The DSL, inherits this "contract" atm.
> From a DSL point of view, it would be desirable to provide a different contract to the user. To allow this, we need to do the following:
>  - extend Processor API to allow manipulation timestamps (ie, a Processor can set a new timestamp for downstream records)
>  - define a DSL "contract" for timestamp propagation for each DSL operator
>  - document the DSL "contract"
>  - implement the DSL "contract" using the new/extended Processor API



--
This message was sent by Atlassian Jira
(v8.3.4#803005)