You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Syinchwun Leo (JIRA)" <ji...@apache.org> on 2017/01/19 13:21:26 UTC
[jira] [Updated] (FLINK-5572) ListState in SlidingEventTimeWindow
and SlidingProcessingTimeWindow optimization
[ https://issues.apache.org/jira/browse/FLINK-5572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Syinchwun Leo updated FLINK-5572:
---------------------------------
Description:
When using ListState in SlidingEventTimeWindow and SlidingProcessingTimeWindow, an element may be assigned to multiple overlapped windows. It may lead to storage consuming. for example,
window(SlidingEventTimeWindows.of(Time.seconds(10), Time.seconds(2))).apply(UDF window function), each element is assigned to 5 windows, When the window size is very large, it is unacceptable(size/slide is very large).
We plan to make a little optimization, and the doc is in
https://docs.google.com/document/d/1HCt1Si3YNGFwsl2H5SO0f7WD69DdBBPVJA6abd3oFWo/edit?usp=sharing
Comments?
was:
When using ListState in SlidingEventTimeWindow and SlidingProcessingTimeWindow, an element may be assigned to multiple overlapped windows. It may lead to storage consuming. for example,
window(SlidingEventTimeWindows.of(Time.seconds(10), Time.seconds(2))).apply(UDF window function), each element is assigned to 5 windows, shown as the following Fig. When the window size is very large, it is unacceptable(size/slide is very large).
We plan to make a little optimization, and the doc is in
https://docs.google.com/document/d/1HCt1Si3YNGFwsl2H5SO0f7WD69DdBBPVJA6abd3oFWo/edit?usp=sharing
Comments?
> ListState in SlidingEventTimeWindow and SlidingProcessingTimeWindow optimization
> --------------------------------------------------------------------------------
>
> Key: FLINK-5572
> URL: https://issues.apache.org/jira/browse/FLINK-5572
> Project: Flink
> Issue Type: Improvement
> Components: Core
> Affects Versions: 1.2.0, 1.1.4
> Environment: CentOS 7.2
> Reporter: Syinchwun Leo
>
> When using ListState in SlidingEventTimeWindow and SlidingProcessingTimeWindow, an element may be assigned to multiple overlapped windows. It may lead to storage consuming. for example,
> window(SlidingEventTimeWindows.of(Time.seconds(10), Time.seconds(2))).apply(UDF window function), each element is assigned to 5 windows, When the window size is very large, it is unacceptable(size/slide is very large).
> We plan to make a little optimization, and the doc is in
> https://docs.google.com/document/d/1HCt1Si3YNGFwsl2H5SO0f7WD69DdBBPVJA6abd3oFWo/edit?usp=sharing
> Comments?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)