[jira] [Commented] (KAFKA-10493) KTable out-of-order updates are not being ignored

["Matthias J. Sax (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-10493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17411592#comment-17411592 ] 

Matthias J. Sax commented on KAFKA-10493:
-----------------------------------------

Agreed. The original API only had `builder.table()` and the assumption was that people would follow the "single writer principle" to avoid out-of-order data in the input topic. When we added `toTable()` the case you describe was actually discussed, as it was a concern... In the end, we decided to add `toTable()` anyway, and leave it to the user's responsibility to be aware of the issue.

In the end, from my POV only "versioned tables" will solve the issue correctly, and we are already working on some design/prototyping.

> KTable out-of-order updates are not being ignored
> -------------------------------------------------
>
>                 Key: KAFKA-10493
>                 URL: https://issues.apache.org/jira/browse/KAFKA-10493
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions: 2.6.0
>            Reporter: Pedro Gontijo
>            Assignee: Matthias J. Sax
>            Priority: Blocker
>             Fix For: 4.0.0
>
>         Attachments: KTableOutOfOrderBug.java, out-of-order-table.png
>
>
> On a materialized KTable, out-of-order records for a given key (records which timestamp are older than the current value in store) are not being ignored but used to update the local store value and also being forwarded.
> I believe the bug is here: [https://github.com/apache/kafka/blob/2.6.0/streams/src/main/java/org/apache/kafka/streams/state/internals/ValueAndTimestampSerializer.java#L77] It should return true, not false (see javadoc)
> The bug impacts here: [https://github.com/apache/kafka/blob/2.6.0/streams/src/main/java/org/apache/kafka/streams/kstream/internals/KTableSource.java#L142-L148]
> I have attached a simple stream app that shows the issue happening.
> Thank you!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)