You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Damien Gasparina (Jira)" <ji...@apache.org> on 2021/07/01 16:37:00 UTC
[jira] [Updated] (KAFKA-13024) Kafka Streams is dropping messages
with null key during repartition
[ https://issues.apache.org/jira/browse/KAFKA-13024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Damien Gasparina updated KAFKA-13024:
-------------------------------------
Description:
{{KStream.repartition}} is silently filtering messages with null keys. A single topology like {{.stream().repartition().to()}} would filter all messages with null key.
The cause: we are adding a filter before the source & sink nodes ([https://github.com/apache/kafka/blob/2.8/streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamImpl.java#L1060-L1064]). It looks like we are doing that because this method is also used for building KTable.
Null key messages are valid for a KStream, it looks like a regression, the previous {{.through()}} was not filtering null key messages.
was:
{{KStream.repartition}} is silently filtering messages with null keys. A single topology like {{.stream().repartition().to()}} would filter all messages with null key.
We are adding a filtering before the repartition ([https://github.com/apache/kafka/blob/2.8/streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamImpl.java#L1060-L1064]). It looks like we are doing that because this method is also used for building KTable.
Null key messages are valid for a KStream, it looks like a regression, the previous {{.through()}} was not filtering null key messages.
> Kafka Streams is dropping messages with null key during repartition
> -------------------------------------------------------------------
>
> Key: KAFKA-13024
> URL: https://issues.apache.org/jira/browse/KAFKA-13024
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 2.8.0, 2.7.1
> Reporter: Damien Gasparina
> Priority: Major
>
> {{KStream.repartition}} is silently filtering messages with null keys. A single topology like {{.stream().repartition().to()}} would filter all messages with null key.
> The cause: we are adding a filter before the source & sink nodes ([https://github.com/apache/kafka/blob/2.8/streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamImpl.java#L1060-L1064]). It looks like we are doing that because this method is also used for building KTable.
> Null key messages are valid for a KStream, it looks like a regression, the previous {{.through()}} was not filtering null key messages.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)