You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Victoria Bialas (Jira)" <ji...@apache.org> on 2019/10/25 17:58:00 UTC
[jira] [Comment Edited] (KAFKA-6453) Reconsider timestamp
propagation semantics
[ https://issues.apache.org/jira/browse/KAFKA-6453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16959956#comment-16959956 ]
Victoria Bialas edited comment on KAFKA-6453 at 10/25/19 5:57 PM:
------------------------------------------------------------------
Helpful links:
* Related KIP-258: [https://cwiki.apache.org/confluence/display/KAFKA/KIP-258%3A+Allow+to+Store+Record+Timestamps+in+RocksDB]
* Earlier KIP (upon which this new work is based) KIP-251: [https://cwiki.apache.org/confluence/display/KAFKA/KIP-251%3A+Allow+timestamp+manipulation+in+Processor+API]
* Docs in Confluent pertinent to WindowStore: [Windowing|#windowing]
was (Author: orangesnap):
Helpful links:
* Related KIP: [KIP-258|[https://cwiki.apache.org/confluence/display/KAFKA/KIP-258%3A+Allow+to+Store+Record+Timestamps+in+RocksDB]]
* Earlier KIP (upon which this new work is based): [KIP-251]([https://cwiki.apache.org/confluence/display/KAFKA/KIP-251%3A+Allow+timestamp+manipulation+in+Processor+API])
* Docs in Confluent pertinent to WindowStore: [Windowing|#windowing]]
> Reconsider timestamp propagation semantics
> ------------------------------------------
>
> Key: KAFKA-6453
> URL: https://issues.apache.org/jira/browse/KAFKA-6453
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Matthias J. Sax
> Assignee: Victoria Bialas
> Priority: Major
> Labels: needs-kip
>
> Atm, Kafka Streams only has a defined "contract" about timestamp propagation at the Processor API level: all processor within a sub-topology, see the timestamp from the input topic record and this timestamp will be used for all result record when writing them to an topic, too.
> The DSL, inherits this "contract" atm.
> From a DSL point of view, it would be desirable to provide a different contract to the user. To allow this, we need to do the following:
> - extend Processor API to allow manipulation timestamps (ie, a Processor can set a new timestamp for downstream records)
> - define a DSL "contract" for timestamp propagation for each DSL operator
> - document the DSL "contract"
> - implement the DSL "contract" using the new/extended Processor API
--
This message was sent by Atlassian Jira
(v8.3.4#803005)