[jira] [Updated] (FLINK-40831) Mini-batch outer join loses the null-padded row when an input record is updated without UPDATE_BEFORE

["Gustavo de Morais (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-40831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gustavo de Morais updated FLINK-40831:
--------------------------------------
    Affects Version/s: 2.1.3
                       1.20.5
                       2.2.1
                       2.3.0

> Mini-batch outer join loses the null-padded row when an input record is updated without UPDATE_BEFORE
> -----------------------------------------------------------------------------------------------------
>
>                 Key: FLINK-40831
>                 URL: https://issues.apache.org/jira/browse/FLINK-40831
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Table SQL / API
>    Affects Versions: 2.3.0, 2.2.1, 1.20.5, 2.1.3
>            Reporter: Gustavo de Morais
>            Assignee: Gustavo de Morais
>            Priority: Major
>
> {{MiniBatchStreamingJoinOperator}} shares the per-record logic of {{{}StreamingJoinOperator{}}}, so it has the same problem. When a join input has no UPDATE_BEFORE, an UPDATE_AFTER can replace a stored record with the same unique key, and the operator counts it as a new match on the outer side. After the record is deleted, the count doesn't reach 0, and the null-padded row is never emitted again.
> Mini-batch adds one more case. Within a bundle, a {{{}-U{}}}/{{{}+U{}}} pair on the same unique key is folded into a suppressed pair: the {{-U}} decrements the count but keeps the record in state. The paired {{+U}} must then always count again. Otherwise an outer row with two matches loses one, and deleting the other match emits a null-padded row while a match still exists.
> Example, with order 1 matching two shipments, {{S1}} and {{{}S2{}}}:
>  # One bundle: {{{}-U S1{}}}, {{+U S1'}} → the count must stay 2.
>  # {{-D S2}} → the count goes to 1. Only {{-D[order 1, S2]}} is expected, with no {{{}+I[order 1, null]{}}}.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)