[jira] [Commented] (KAFKA-8504) Suppressed do not emit with TimeWindows

["Matthias J. Sax (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-8504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16859341#comment-16859341 ] 

Matthias J. Sax commented on KAFKA-8504:
----------------------------------------

While I agree, that it would be better to use a default value for zero for grace-period, it's not easily possible, due do backward compatibility issues. Compare the KIP for more details: [https://cwiki.apache.org/confluence/display/KAFKA/KIP-328%3A+Ability+to+suppress+updates+for+KTables]

In order versions, there was only "retention time" parameter that is set to 1 day by default. To be backward compatible, we need to set grace period to "-1". For this case, retention-time parameter is interpreted as grace period (roughly), implying that grace period is effectively 1 day by default.

Hence, if you use suppress() with default values, it won't emit any final result for 24h. In practices, you would need to set grace-period to a smaller value, depending on your application requirements.

I was just double checking the docs, and it seems this behavior is actually not well documents (neither on AK webpage nor in JavaDocs). I think we should improve the docs accordingly.

 

> Suppressed do not emit with TimeWindows
> ---------------------------------------
>
>                 Key: KAFKA-8504
>                 URL: https://issues.apache.org/jira/browse/KAFKA-8504
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions: 2.2.1
>            Reporter: Simone
>            Priority: Minor
>
> Hi, I'm playing a bit with KafkaStream and the new suppress feature. I noticed that when using a {{TimeWindows}} without explicitly setting the grace {{suppress}} will not emit any message if used with {{Suppressed.untilWindowCloses.}}
> I look a bit into the code and from what I understood with this configuration {{suppress}} should use the {{grace}} setting of the {{TimeWindows}}. But since using {{TimeWindows.of(Duration)}} default the grace to {{-1}} and when getting the grace using the method {{TimeWindows.gracePeriodMs()}} in case of grace equals to -1 the return value is set to {{maintainMs() - size()}} I think that the end of window is not properly calculated. 
> Of course is possible to avoid this problem forcing the {{grace}} to 0 when creating the TimeWindows but I think that this should be the default behaviour at least when it comes to the suppress feature.
> I hope I have not misunderstood the code in my analysis, thank you :)
> Simone



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)