You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "fightBoxing (via GitHub)" <gi...@apache.org> on 2026/08/18 07:52:18 UTC

[PR] [oracle] Support specific-offset startup mode with SCN for Oracle pipeline source [flink-cdc]

fightBoxing opened a new pull request, #4510:
URL: https://github.com/apache/flink-cdc/pull/4510

   ## Motivation
   
   The Oracle pipeline connector currently only supports `initial` / `snapshot` / `latest-offset` startup modes, while the MySQL pipeline connector already supports `specific-offset`. For Oracle users who want to skip the full snapshot phase and start incremental (LogMiner redo log) consumption from a known position, there is no way to specify a starting SCN.
   
   Note: the underlying Debezium property `log.mining.start.position.scn` cannot be used as a workaround, because the shaded Debezium dependency in the connector jar does not include that configuration, and passing it through the `debezium.*` prefix is silently ignored.
   
   ## Modification
   
   - `OracleDataSourceOptions`: add new option `scan.startup.specific-offset.scn` (Long, no default value)
   - `OracleDataSourceFactory`: add `specific-offset` branch in `getStartupOptions()`, which builds `StartupOptions.specificOffset(Map{scn, commit_scn})`. The base framework (`StreamSplitAssigner#createStreamSplit`) already natively consumes SPECIFIC_OFFSETS offsets, so no framework change is needed.
   
   ## Usage
   
   ```yaml
   source:
     type: oracle
     scan.startup.mode: specific-offset
     scan.startup.specific-offset.scn: 2053676
   ```
   
   ## Verification
   
   Verified end-to-end on a real cluster (Flink 1.20.1 YARN session + Oracle 12c LogMiner + Iceberg sink):
   
   - JobManager log confirms the stream split starts exactly at the specified SCN: `Assign split StreamSplit{offset={commit_scn=0, scn=2053676}}`
   - Only changes committed after the specified SCN are captured; all pre-existing snapshot rows are correctly skipped


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org