[jira] [Created] (FLINK-39030) Support emit-only-on-update mode for CUMULATE window TVF to reduce unnecessary outputs

["Liu (Jira)" <ji...@apache.org>]

Liu created FLINK-39030:
---------------------------

             Summary: Support emit-only-on-update mode for CUMULATE window TVF to reduce unnecessary outputs
                 Key: FLINK-39030
                 URL: https://issues.apache.org/jira/browse/FLINK-39030
             Project: Flink
          Issue Type: Improvement
          Components: Table SQL / API
            Reporter: Liu


h1. Summary

Currently, the CUMULATE window TVF (Table-Valued Function) triggers output at every step interval, regardless of whether new data arrived during that period. This behavior leads to unnecessary repeated outputs and increased downstream processing overhead in scenarios where data arrives sparsely.
This proposal introduces an optional emit-only-on-update mode for CUMULATE windows that only emits results when new data actually arrives within a step interval.
h1. Motivation

Consider a CUMULATE window with maxSize=15 seconds and step=5 seconds, starting at 00:00:00:

 
||Window||Time range||Triggered at||
|1|[00:00:00, 00:00:05)|00:00:05|
|2|[00:00:00, 00:00:10)|00:00:10|
|3|[00:00:00, 00:00:15)|00:00:15|


*Problem:* If data only arrives at 00:00:02, the current implementation still emits three outputs (at 00:00:05, 00:00:10, and 00:00:15), even though windows 2 and 3 contain no new data compared to window 1.
*Impact:*

 
 * Unnecessary computational overhead
 * Increased network I/O for downstream operators
 * Higher storage costs when persisting results
 * Significant performance degradation in scenarios with sparse data distribution

h1. Proposed Solution

Add an optional 6th parameter emit_on_update_only (BOOLEAN, default false) to the CUMULATE TVF:
{code:java}
CUMULATE(
    TABLE data_table,
    DESCRIPTOR(rowtime),
    INTERVAL '5' SECOND,    -- step
    INTERVAL '15' SECOND,   -- maxSize  
    INTERVAL '0' SECOND,    -- offset
    true                    -- emit_on_update_only (NEW)
) {code}
Behavior when emit_on_update_only = true:
 * Window output is only triggered when new data arrives within the current step interval
 * If no new data arrives, the window silently advances without emitting results
 * This is particularly useful for sparse data streams where most step intervals have no data

Behavior when emit_on_update_only = false (default):
 * Current behavior is preserved (output at every step interval)
 * Ensures backward compatibility



--
This message was sent by Atlassian Jira
(v8.20.10#820010)