You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@fluss.apache.org by "luoyuxia (via GitHub)" <gi...@apache.org> on 2026/03/16 09:40:28 UTC
[PR] [flink] Fix $binlog update pairing across log splits [fluss]
luoyuxia opened a new pull request, #2880:
URL: https://github.com/apache/fluss/pull/2880
### Purpose
Linked issue: close #2879
This PR backports the `$binlog` update-pairing fix to `release-0.9`.
When one Flink `SourceReader` processes records from multiple log splits, `$binlog`
can interleave `UPDATE_BEFORE` / `UPDATE_AFTER` records from different splits.
The previous implementation kept only one global pending `UPDATE_BEFORE` record,
which could be overwritten by another split and then fail with:
```text
Received UPDATE_AFTER (+U) without a preceding UPDATE_BEFORE (-U) record.
```
### Brief change log
- keep pending `UPDATE_BEFORE` state per split instead of using one global buffer;
- pass the current split id through the emitter/deserialization path for `$binlog`;
- pair `UPDATE_BEFORE` and `UPDATE_AFTER` only within the same split;
- add a regression test for cross-split interleaving in `BinlogRowConverterTest`.
### Tests
- Added `BinlogRowConverterTest#testCrossSplitInterleavingDoesNotCorrupt`
- Not run locally
### API and Format
No API or storage format changes.
### Documentation
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [flink] Fix $binlog update pairing across log splits [fluss]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #2880:
URL: https://github.com/apache/fluss/pull/2880#discussion_r2939244820
##########
fluss-flink/fluss-flink-common/src/main/java/org/apache/fluss/flink/utils/BinlogRowConverter.java:
##########
@@ -72,13 +86,14 @@ public RowData toBinlogRowData(LogRecord record) {
baseConverter.toFlinkRowData(record.getRow()));
case UPDATE_BEFORE:
- // Buffer the -U record and return null.
+ // Buffer the -U record per split and return null.
// FlinkRecordEmitter.processAndEmitRecord() skips null results.
Review Comment:
In UPDATE_BEFORE handling, `pendingUpdateBeforeMap.put(splitId, record)` will silently overwrite an existing buffered -U for the same splitId (e.g., if two UPDATE_BEFORE records arrive before an UPDATE_AFTER due to a corrupted sequence or duplicate delivery). That would cause the subsequent +U to merge with the wrong before-image instead of failing fast. Consider detecting an existing pending entry for the split and throwing an IllegalStateException (or otherwise handling it explicitly) to avoid producing incorrect $binlog output.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [flink] Fix $binlog update pairing across log splits [fluss]
Posted by "luoyuxia (via GitHub)" <gi...@apache.org>.
luoyuxia closed pull request #2880: [flink] Fix $binlog update pairing across log splits
URL: https://github.com/apache/fluss/pull/2880
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org