[jira] [Created] (FLINK-34694) Delete num of associations for streaming outer join

["Roman Boyko (Jira)" <ji...@apache.org>]

Roman Boyko created FLINK-34694:
-----------------------------------

             Summary: Delete num of associations for streaming outer join
                 Key: FLINK-34694
                 URL: https://issues.apache.org/jira/browse/FLINK-34694
             Project: Flink
          Issue Type: Improvement
          Components: Table SQL / Runtime
    Affects Versions: 1.18.1, 1.19.0, 1.17.2, 1.16.3
            Reporter: Roman Boyko
         Attachments: image-2024-03-15-19-51-29-282.png, image-2024-03-15-19-52-24-391.png

Currently in StreamingJoinOperator (non-window) in case of OUTER JOIN the OuterJoinRecordStateView is used to store additional field - the number of associations for every record. This leads to store additional Tuple2 and Integer data for every record in outer state.

This functionality is used only for sending:
 * -D[nullPaddingRecord] in case of first Accumulate record
 * +I[nullPaddingRecord] in case of last Revoke record

The overhead of storing additional data and updating the counter for associations can be avoided by checking the input state for these events.

 

The proposed solution can be found here - [https://github.com/rovboyko/flink/commit/1ca2f5bdfc2d44b99d180abb6a4dda123e49d423]

 

According to the nexmark q20 test (changed to OUTER JOIN) it could increase the performance up to 20%:
 * Before:

!image-2024-03-15-19-52-24-391.png!
 * After:

!image-2024-03-15-19-51-29-282.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)