You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "li3zhi4 (via GitHub)" <gi...@apache.org> on 2026/08/01 07:32:30 UTC

[PR] [Fix][Connector-CDC] Implement MySQL CDC stop.mode = "specific" (bounded read) [seatunnel]

li3zhi4 opened a new pull request, #11618:
URL: https://github.com/apache/seatunnel/pull/11618

   ## Why
   
   `stop.mode = "specific"` (and by the same code path `"latest"`) is documented but never works for MySQL CDC:
   the incremental split is read by `MySqlBinlogFetchTask`, which creates a plain Debezium
   `MySqlStreamingChangeEventSource` with **no stopOffset check**, so the job runs forever even after
   the configured stop position has been passed. Previously reported in #6789 and #7400; both were
   auto-closed by the stale bot without a fix.
   
   This PR implements bounded read for the MySQL incremental split so the job terminates at the exact
   configured binlog offset, plus two related correctness fixes found while debugging it.
   
   Closes #11617
   
   ## What changed
   
   ### 1. Bounded read for incremental split (`stop.mode = "specific"`)
   
   - `connector-cdc-base/.../source/offset/Offset.java`
     - Add `isNeverStop()` default implementation (subclasses override with their unbounded sentinel).
   - `connector-cdc-mysql/.../source/reader/fetch/binlog/MySqlBinlogFetchTask.java`
     - Add `BoundedMySqlStreamingChangeEventSource` inner class: checks the split's stop offset on
       every event, ends the binlog read when the target position is reached, and logs binlog progress
       periodically. The unbounded (`stop.mode = "never"`) path is unchanged.
   - `connector-cdc-base/.../source/reader/IncrementalSourceReader.java`
     - Allow incremental splits to finish normally (previously only snapshot splits could finish).
   - `connector-cdc-base/.../source/reader/external/IncrementalSourceStreamFetcher.java`
     - `pollSplitRecords()` returns `null` to signal split completion **only for bounded reads**
       (`!stopOffset.isNeverStop()`), so unbounded jobs can never be falsely marked FINISHED.
   
   ### 2. Fix `BinlogOffset.compareTo()` when GTID sets are mixed
   
   - `connector-cdc-mysql/.../source/offset/BinlogOffset.java`
     - Correct the GTID fallback comparison so offsets with mixed/partial GTID sets compare correctly
       (needed by both start-offset and stop-offset comparisons).
   
   ### 3. Fix `taskStarted` race in `IncrementalSourceStreamFetcher`
   
   - Set `taskStarted`/`executing` synchronously when submitting the task, closing a race window that
     could make a bounded-read job finish prematurely with silently missing data (affects
     `stop.mode = "specific"` and `"latest"`, not `"never"`).
   
   ### 4. Minor: correct `stop.mode` descriptions
   
   - Oracle / SQL Server CDC option descriptions updated to match the actual supported value `"never"`.
   
   ## Verification
   
   - Manual test with `stop.mode = "specific"` (start `mysql-bin.059734:4`, stop `mysql-bin.059818:4`)
     on a production-like table: job now terminates with `FINISHED` exactly at the stop position
     (previously ran forever).
   - `stop.mode = "never"` (default) regression: unbounded job keeps running; split-completion `null`
     path is not triggered.
   - `mvn install -pl connector-cdc/connector-cdc-mysql -am -DskipTests` passes.
   
   ## Checklist
   
   - [x] Code changes complete and verified (ported from 2.3.13 custom branch, rebased onto dev layout)
   - [x] Related issue referenced: #11617
   - [x] Compiles with `mvn install -DskipTests` for the CDC module chain
   - [ ] `spotless:check` (full CI will verify)
   - [ ] Unit test for `BinlogOffset.compareTo()` mixed-GTID cases (follow-up)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-CDC] Implement MySQL CDC stop.mode = "specific" (bounded read) [seatunnel]

Posted by "li3zhi4 (via GitHub)" <gi...@apache.org>.
li3zhi4 commented on PR #11618:
URL: https://github.com/apache/seatunnel/pull/11618#issuecomment-5164651247

   Thanks for the sharp catch, @nielifeng. You're right — `isNeverStop()` was only overridden in MySQL's `BinlogOffset`, so the other CDC connectors' `NO_STOPPING_OFFSET` sentinels were classified as bounded by the shared reader, which could silently FINISH an unbounded job. I've implemented your option 1 (commit `cd0274ee7d`):
   
   **`isNeverStop()` implemented for every sentinel offset used by the shared reader:**
   - `LsnOffset` (PostgreSQL + SQL Server): `NO_STOPPING_OFFSET.equals(this)`
   - `RedoLogOffset` (Oracle): `NO_STOPPING_OFFSET.equals(this)`
   - `ChangeStreamOffset` (MongoDB): `NO_STOPPING_OFFSET.equals(this)`
   
   **Regression tests:**
   - Per-connector sentinel tests (`LsnOffsetTest` PG/SQLServer, `RedoLogOffsetTest`, `ChangeStreamOffsetTest`) asserting `NO_STOPPING_OFFSET.isNeverStop()` is `true` and a normal offset is not
   - `IncrementalSourceStreamFetcherTest.testNeverStopSentinelIsNotSignaledAsBoundedCompletion`: simulates a finished fetch task with a real never-stop sentinel and asserts `pollSplitRecords()` returns an **empty iterator instead of null** — i.e. an unbounded connector is never marked completed by the shared reader
   
   **Verification (all local):**
   - `spotless:check` ✅ (cdc-base + 4 connector modules)
   - `IncrementalSourceStreamFetcherTest` 3/3 ✅, `LsnOffsetTest` 2/2 (each) ✅, `RedoLogOffsetTest` 1/1 ✅, `ChangeStreamOffsetTest` 1/1 ✅
   - compile ✅
   
   Happy to adjust if you'd prefer the explicit split/config contract (option 2) instead.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-CDC] Implement MySQL CDC stop.mode = "specific" (bounded read) [seatunnel]

Posted by "li3zhi4 (via GitHub)" <gi...@apache.org>.
li3zhi4 commented on PR #11618:
URL: https://github.com/apache/seatunnel/pull/11618#issuecomment-5163736493

   Hi @DanielLeens, I've addressed both non-blocking suggestions from your re-review (commit `dec6fecc3e`):
   
   **Suggestion A — regression coverage for `startup.mode = timestamp + stop.mode = specific`:**
   - New e2e `MysqlCDCStopModeSpecificIT.testMysqlCdcStopModeSpecificWithTimestampStartup` (new conf `mysqlcdc_stop_mode_specific_timestamp.conf`)
   - Asserts: rows written before the startup timestamp are **not** synced, rows between the startup timestamp and the stop offset **are** synced, rows after the stop offset are **not** synced, and the job **terminates (FINISHED)** at the configured stop offset
   - Locally verified: `Tests run: 2, Failures: 0, Errors: 0` (both `specific+specific` and `timestamp+specific` methods pass)
   
   **Suggestion B — bounded-read docs example:**
   - Added a "Bounded Read: Stop at a Specific Binlog Offset" section to both `docs/en/connectors/source/MySQL-CDC.md` and `docs/zh/connectors/source/MySQL-CDC.md`, with examples for `stop.mode=specific` alone and combined with `startup.mode=timestamp`
   
   Full local verification for this head: `spotless:check` ✅, `IncrementalSourceStreamFetcherTest` 2/2 ✅, `MysqlCDCStopModeSpecificIT` 2/2 ✅, compile ✅.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-CDC] Implement MySQL CDC stop.mode = "specific" (bounded read) [seatunnel]

Posted by "li3zhi4 (via GitHub)" <gi...@apache.org>.
li3zhi4 commented on PR #11618:
URL: https://github.com/apache/seatunnel/pull/11618#issuecomment-5159431390

   Thanks for the detailed re-review, @DanielLeens. I've fixed the tail-data-loss race in the completion handshake (commit `5b088374e5`):
   
   **What changed in `IncrementalSourceStreamFetcher`:**
   - `pollSplitRecords()` now **always drains the queue first**; split completion (`return null`) is only signaled after the queue is empty AND the bounded split is finished. Previously it returned `null` before draining, which could drop the last queued batch.
   - `splitNormalStream()` guard now accepts the bounded-finished state (`isRunning() || isBoundedReadFinished()`), so the final batch is emitted even after `isRunning()` flipped false — previously the second `isRunning()` guard could filter out a just-drained tail batch.
   - Extracted `isBoundedReadFinished()` helper shared by both places.
   
   **New unit test** `testPollSplitRecordsDrainsQueueBeforeBoundedCompletion`: simulates a finished bounded fetch task with one last batch still in the queue — asserts the batch is returned first, and only the subsequent poll (empty queue) returns `null`.
   
   **Verification (all local):**
   - `spotless:check` ✅
   - `IncrementalSourceStreamFetcherTest` → `Tests run: 2, Failures: 0` ✅
   - `MysqlCDCStopModeSpecificIT` e2e (Docker, Zeta engine) → `Tests run: 1, Failures: 0, Errors: 0` ✅ — the e2e already asserts the pre-stop-offset rows (21/22) are synced, which covers exactly this tail-batch scenario
   - Branch is up to date with `dev` (merged `1b4d6c2ef8`)
   
   I believe all previously raised points are now addressed. Happy to iterate further if anything else comes up.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-CDC] Implement MySQL CDC stop.mode = "specific" (bounded read) [seatunnel]

Posted by "li3zhi4 (via GitHub)" <gi...@apache.org>.
li3zhi4 commented on PR #11618:
URL: https://github.com/apache/seatunnel/pull/11618#issuecomment-5191132414

   Thanks for the thorough re-review, @DanielLeens. Addressing the two items from your latest round:
   
   **Issue 2 (docs, commit `f6a18f07df`):** Added a one-line engine caveat to both the English and Chinese MySQL-CDC bounded-read sections, noting that `stop.mode="specific"` termination is currently supported on the **Zeta** engine only (Flink/Spark do not support bounded incremental-split termination yet, matching the `@DisabledOnContainer` restriction in the e2e test).
   
   **Issue 1 (CI):** I re-ran the failed jobs on the fork (`gh run rerun --failed` on the current head's Build run) — the failing jobs (part-4, part-6, unit-test windows, kafka-connector-it) are all unrelated to this PR's diff, consistent with your log analysis. Once the rerun completes, the aggregated Build check should go green.
   
   Note: the checkpoint/restore regression test for the bounded-read path (your recommended, non-blocking item) is not included in this push — happy to add it as a follow-up if you'd like.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-CDC] Implement MySQL CDC stop.mode = "specific" (bounded read) [seatunnel]

Posted by "li3zhi4 (via GitHub)" <gi...@apache.org>.
li3zhi4 commented on PR #11618:
URL: https://github.com/apache/seatunnel/pull/11618#issuecomment-5157328038

   Thanks for the thorough review, @DanielLeens. I have addressed all three blockers:
   
   **Blocker 1 (timestamp + specific combination):** `MySqlBinlogFetchTask.execute()` now routes `startup.mode=timestamp` + `stop.mode=specific` into `BoundedMySqlStreamingChangeEventSource`, which accepts an optional timestamp filter. `handleEvent()` skips events before the target timestamp and still checks the stop offset in the `finally` block, so the job terminates at the configured binlog offset instead of running forever.
   
   **Blocker 2 (Oracle regression):** Reverted the Oracle/SqlServer option-rule changes from this PR (`[Revert][Connector-CDC] Restore Oracle/SqlServer option rules to upstream state`). Oracle `STARTUP_MODE` is back to `INITIAL/LATEST/TIMESTAMP` with the upstream description — the diff for both files is now empty against `dev`.
   
   **Blocker 3 (Code style):** Ran `spotless:apply` on `BinlogOffsetTest.java`; `spotless:check` now passes locally for the module.
   
   Also verified locally: `mvn install -pl connector-cdc/connector-cdc-mysql -am -DskipTests` → BUILD SUCCESS, and `BinlogOffsetTest` 9/9 tests pass.
   
   Non-blocking suggestions (docs example, log dedup) I'll keep as follow-ups to keep this PR single-purpose.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-CDC] Implement MySQL CDC stop.mode = "specific" (bounded read) [seatunnel]

Posted by "li3zhi4 (via GitHub)" <gi...@apache.org>.
li3zhi4 commented on PR #11618:
URL: https://github.com/apache/seatunnel/pull/11618#issuecomment-5157574289

   Hi @SEZ9, per your request I've added an e2e test covering the `stop.mode="specific"` termination behavior:
   
   **New: `MysqlCDCStopModeSpecificIT`** (commit `de40a3d6f0`)
   - Starts a MySQL CDC job with `startup.mode=specific` + `stop.mode=specific` (new conf `mysqlcdc_stop_mode_specific.conf`)
   - Records the current binlog offset as the stop offset, writes rows before and after it
   - Asserts the job **terminates on its own (FINISHED)** at the configured stop offset instead of running forever (`jobFuture.get(120s)` with exit code 0)
   - Verifies only the rows written before the stop offset are synced (rows after are not)
   
   Also, regarding your note to check other CDC connectors sharing this fetch-task pattern: the same `FetchTask`-based bounded-read issue applies to any CDC source whose `MySqlBinlogFetchTask`-equivalent creates a plain streaming source without a stop check. This PR fixes the MySQL path; Oracle/SqlServer/Postgres only support `stop.mode=never` in their option rules today, so they cannot hit the combination. I can file a follow-up issue to audit the other connectors if the community wants.
   
   Verification: `spotless:check` passes; the new test class compiles. I could not run the e2e locally (no Docker in this environment), so it will be exercised by the CI e2e job.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-CDC] Implement MySQL CDC stop.mode = "specific" (bounded read) [seatunnel]

Posted by "SEZ9 (via GitHub)" <gi...@apache.org>.
SEZ9 commented on PR #11618:
URL: https://github.com/apache/seatunnel/pull/11618#issuecomment-5225134635

   Thanks @li3zhi4 for the quick turnaround! 👍
   
   The new `MysqlCDCStopModeSpecificIT` looks like a solid coverage approach — asserting the job reaches FINISHED at the recorded stop offset and verifying only pre-offset rows are synced is exactly the behavior we want to lock in.
   
   A few remaining items:
   
   1. **CI verification**: Since you couldn't run the e2e locally, let's wait for the CI e2e job to pass before merging. Please keep an eye on the run and ping me if anything flaky shows up.
   2. **Follow-up issue**: Yes, please do file a follow-up issue to audit the other `FetchTask`-based CDC connectors. Even though Oracle/SqlServer/Postgres only expose `stop.mode=never` in their option rules today, having the audit tracked will help whoever extends stop-mode support there later. Feel free to reference this PR in the issue.
   3. Minor ask: could you double-check the 120s timeout in `jobFuture.get(120s)` is generous enough for the CI environment? Our CI runners can be slower than local machines, and we've seen CDC e2e tests need extra headroom for container startup + binlog replication.
   
   Otherwise this looks good to me. Once CI is green and the follow-up issue is filed, I'm happy to move forward.
   
   <!-- streview-comment:84 -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-CDC] Implement MySQL CDC stop.mode = "specific" (bounded read) [seatunnel]

Posted by "davidzollo (via GitHub)" <gi...@apache.org>.
davidzollo commented on PR #11618:
URL: https://github.com/apache/seatunnel/pull/11618#issuecomment-5192167828

   Thanks for your contribution! This is a solid addition to the project, and your attention to detail really shows. Contributors like you are what keep this community moving forward. Feel free to keep the PRs coming — we're always glad to have you here!
   
   Please ignore the following message if you have joined in the Slack~
   ```
   To facilitate focused communication among contributors, we’ve created a brand-new dedicated #contributors channel in official Slack workspace.
   Slack workspace link: https://s.apache.org/seatunnel-slack
   
   This channel is designed for technical discussions, feature design brainstorming, roadmap sync and direct communication with project core Contributors. Everyone who contributes code, docs, test cases or community ideas is warmly invited to join, share thoughts and collaborate together under The Apache Way. Welcome to join 😄
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-CDC] Implement MySQL CDC stop.mode = "specific" (bounded read) [seatunnel]

Posted by "DanielLeens (via GitHub)" <gi...@apache.org>.
DanielLeens commented on PR #11618:
URL: https://github.com/apache/seatunnel/pull/11618#issuecomment-5231608262

   @/private/tmp/claude-501/-Volumes-disk6-tmp-seatunnel-third-nervous-swirles-887e0f/f978b90c-4e26-438b-9ad4-2fbf1f56ae20/scratchpad/pr-batch-20260809/post_bodies/reply_11618.txt


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-CDC] Implement MySQL CDC stop.mode = "specific" (bounded read) [seatunnel]

Posted by "li3zhi4 (via GitHub)" <gi...@apache.org>.
li3zhi4 commented on PR #11618:
URL: https://github.com/apache/seatunnel/pull/11618#issuecomment-5159362742

   Good news: I've now **actually run the new e2e test locally** (with Docker) and it passes.
   
   **Result:** `MysqlCDCStopModeSpecificIT` → `Tests run: 1, Failures: 0, Errors: 0` ✅ (BUILD SUCCESS)
   
   What the test verifies end-to-end:
   1. **Termination**: starts a MySQL CDC job with `startup.mode=specific` + `stop.mode=specific`, and asserts the job **terminates on its own (FINISHED)** at the configured binlog offset instead of running forever
   2. **Data boundary**: rows written before the stop offset are synced; rows written after the stop offset are **not** synced (the test inserts them only after the snapshot phase completes, so only the binlog phase could pick them up — and it correctly stops at the stop offset)
   
   Two refinements made while running it locally (commits `de40a3d6f0` + `05f8a0702d`):
   - Test is restricted to the **Zeta engine** via `@DisabledOnContainer` (FLINK/SPARK don't support bounded incremental-split termination; same pattern as the existing `MysqlCDCSpecificStartingOffsetIT`)
   - The post-stop-offset rows are inserted only after the pre-stop rows are confirmed synced, fixing a timing issue where the snapshot phase read all rows
   
   Local environment note: I ran it with `-Dtestcontainer.version=1.21.4` (the repo's pinned 1.17.6 bundles an older docker-java that can't talk to Docker 29's API); CI uses the pinned version. Also merged the latest `dev` into the branch to keep it green.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-CDC] Implement MySQL CDC stop.mode = "specific" (bounded read) [seatunnel]

Posted by "davidzollo (via GitHub)" <gi...@apache.org>.
davidzollo merged PR #11618:
URL: https://github.com/apache/seatunnel/pull/11618


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org