You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Bowen Li (JIRA)" <ji...@apache.org> on 2018/01/09 06:07:00 UTC
[jira] [Commented] (FLINK-3089) State API Should Support Data
Expiration (State TTL)
[ https://issues.apache.org/jira/browse/FLINK-3089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16317800#comment-16317800 ]
Bowen Li commented on FLINK-3089:
---------------------------------
[~srichter] [~aljoscha] guys, I want to follow up on this ticket.
To enforce TtlDB delete all records that have expired before checkpoints/savepoints, we can call {{RocksDB#compactRange()}} (synchronous call) to do a manual compaction.
What are your thoughts?
> State API Should Support Data Expiration (State TTL)
> ----------------------------------------------------
>
> Key: FLINK-3089
> URL: https://issues.apache.org/jira/browse/FLINK-3089
> Project: Flink
> Issue Type: New Feature
> Components: DataStream API, State Backends, Checkpointing
> Reporter: Niels Basjes
> Assignee: Bowen Li
>
> In some usecases (webanalytics) there is a need to have a state per visitor on a website (i.e. keyBy(sessionid) ).
> At some point the visitor simply leaves and no longer creates new events (so a special 'end of session' event will not occur).
> The only way to determine that a visitor has left is by choosing a timeout, like "After 30 minutes no events we consider the visitor 'gone'".
> Only after this (chosen) timeout has expired should we discard this state.
> In the Trigger part of Windows we can set a timer and close/discard this kind of information. But that introduces the buffering effect of the window (which in some scenarios is unwanted).
> What I would like is to be able to set a timeout on a specific state which I can update afterwards.
> This makes it possible to create a map function that assigns the right value and that discards the state automatically.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)