You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Anil (Jira)" <ji...@apache.org> on 2022/05/04 05:54:00 UTC
[jira] [Created] (KAFKA-13870) support both Suppressed untilTimeLimit and maxBytes without using emitEarlyWhenFull()
Anil created KAFKA-13870:
----------------------------
Summary: support both Suppressed untilTimeLimit and maxBytes without using emitEarlyWhenFull()
Key: KAFKA-13870
URL: https://issues.apache.org/jira/browse/KAFKA-13870
Project: Kafka
Issue Type: New Feature
Reporter: Anil
My use case is to use ** *untilTimeLimit* with *maxBytes,* but when the buffer is full, the application is breaking, but with using *{{emitEarlyWhenFull}}* {{{}application is not breaking but{}}}{*}{{}}{*} it sends out the same key record multiple times in a particular window when the buffer exceeds max bytes
for eg:-
*Suppressed.untilTimeLimit(Duration.ofMinutes(15),Suppressed.BufferConfig.maxBytes(10000).emitEarlyWhenFull())*
messages flow : (A,1) (A,2) (A,3) -> aggregation result : (A,6) . suppose here, the buffer is full, (A,6) will be sent downstream. Let's suppose (A,4) comes now in the same tumbling window.
current response:- the aggregation will continue and eventually *(A,10)* will be emitted
but our application expected *(A,4),* so the request for the feature is that window should be happening with window time(untilTimeLimit) or Buffer(maxByte) should full, in either of these two conditions met, a new window should be created and data should be emitted
--
This message was sent by Atlassian Jira
(v8.20.7#820007)