[jira] [Commented] (KAFKA-12960) WindowStore and SessionStore do not enforce strict retention time

["Matthias J. Sax (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-12960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17433856#comment-17433856 ] 

Matthias J. Sax commented on KAFKA-12960:
-----------------------------------------

I just got a report, that `InMemoryWindowStore` does not enforce strict retention for IQ if caching is enabled. We should add corresponding tests before we close this ticket, to ensure that caching does not break strict retention time.

> WindowStore and SessionStore do not enforce strict retention time
> -----------------------------------------------------------------
>
>                 Key: KAFKA-12960
>                 URL: https://issues.apache.org/jira/browse/KAFKA-12960
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Matthias J. Sax
>            Assignee: Sagar Rao
>            Priority: Major
>
> WindowedStore and SessionStore do not implement a strict retention time in general. We should consider to make retention time strict: even if we still have some record in the store (due to the segmented implementation), we might want to filter expired records on-read. This might benefit PAPI users.
> Atm, InMemoryWindow store does already enforce a strict retention time.
> As an alternative, we could also inject such a filter in the wrapping `MeteredStore` – this might lift the burden from users who implement a custom state store.
> As an alternative, we could change all DSL operators to verify if data from a state store is already expired or not. It might be better to push this responsibility into the stores though.
> It's especially an issue for stream-stream joins, because the operator relies on the retention time to implement it's grace period.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)