You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Matthias J. Sax (Jira)" <ji...@apache.org> on 2020/09/18 21:22:00 UTC
[jira] [Commented] (KAFKA-10493) Ktable out-of-order updates are
not being ignored
[ https://issues.apache.org/jira/browse/KAFKA-10493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17198538#comment-17198538 ]
Matthias J. Sax commented on KAFKA-10493:
-----------------------------------------
Not sure if I fully understand (but the code you refer to with regard to idempotent updates is fairly new...).
In general though, KTables don't drop out-of-order updates, but only log a WARN message: cf [https://github.com/apache/kafka/blob/2.6.0/streams/src/main/java/org/apache/kafka/streams/kstream/internals/KTableSource.java#L134-L137]
So it seems to me, that you describe behavior that is by design?
There was a discussion about dropping out-of-order updates and the decision was to not drop them due to backward compatibility concerns. Personally, I agree that we might want to drop out-of-order updates, but we need a clean (ie, opt-in) path forward to change the behavior.
> Ktable out-of-order updates are not being ignored
> -------------------------------------------------
>
> Key: KAFKA-10493
> URL: https://issues.apache.org/jira/browse/KAFKA-10493
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 2.6.0
> Reporter: Pedro Gontijo
> Priority: Major
> Attachments: KTableOutOfOrderBug.java
>
>
> On a materialized KTable, out-of-order records for a given key (records which timestamp are older than the current value in store) are not being ignored but used to update the local store value and also being forwarded.
> I believe the bug is here: [https://github.com/apache/kafka/blob/2.6.0/streams/src/main/java/org/apache/kafka/streams/state/internals/ValueAndTimestampSerializer.java#L77] It should return true, not false (see javadoc)
> The bug impacts here: [https://github.com/apache/kafka/blob/2.6.0/streams/src/main/java/org/apache/kafka/streams/kstream/internals/KTableSource.java#L142-L148]
> I have attached a simple stream app that shows the issue happening.
> Thank you!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)