[jira] [Created] (FLINK-40353) Leverage RocksDB ordered MapState TemporalRowTimeJoinOperator

["Piotr Nowojski (Jira)" <ji...@apache.org>]

Piotr Nowojski created FLINK-40353:
--------------------------------------

             Summary: Leverage RocksDB ordered MapState TemporalRowTimeJoinOperator
                 Key: FLINK-40353
                 URL: https://issues.apache.org/jira/browse/FLINK-40353
             Project: Flink
          Issue Type: Improvement
          Components: Table SQL / Runtime
            Reporter: Piotr Nowojski


Some state backends (e.g. RocksDB) always iterate a map state's keys in a well-defined order (their serialized/binary representation); others (e.g. the heap backend) don't. For example TemporalRowTimeJoinOperator could benefit from ordered iteration have currently no way today to detect this at runtime. 

TemporalRowTimeJoinOperator (row-time FOR SYSTEM_TIME AS OF temporal joins) does a full scan of its left/probe-side state plus a fresh sort of its right/build-side state on every watermark-triggered timer firing, regardless of how many rows actually need processing on that firing. For a join with substantial buffered state, this is a recurring, cost that can be avoided once we expose the information that the underlying state is binary ordered.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)