[jira] [Created] (FLINK-37153) Monitor late event count in temporal join

["Grzegorz Kołakowski (Jira)" <ji...@apache.org>]

Grzegorz Kołakowski created FLINK-37153:
-------------------------------------------

             Summary: Monitor late event count in temporal join
                 Key: FLINK-37153
                 URL: https://issues.apache.org/jira/browse/FLINK-37153
             Project: Flink
          Issue Type: Improvement
          Components: Table SQL / API
            Reporter: Grzegorz Kołakowski


In temporal join, events on the right side (build side, versioned table) are not discarded even if they arrive late (event timestamp < current watermark), which is generally good.

As far as I understand, late events can sometimes lead to incorrect results. This happens when the latest right-side event is delayed for some reason, and as the watermark progresses, a left-side event gets joined with the most recently known right-side event (stored in Flink's state), which might not be the true latest one.

I believe it would be valuable to have a metric tracking late event count on the right side, just to be aware if such situation might have occurred.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)