You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Yanquan Lv (Jira)" <ji...@apache.org> on 2026/09/22 02:42:00 UTC

[jira] [Resolved] (FLINK-40148) TableIdRouter resolves wrong sink table when source regex has multi-digit suffix and sink-table has no back-reference

     [ https://issues.apache.org/jira/browse/FLINK-40148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yanquan Lv resolved FLINK-40148.
--------------------------------
      Assignee: peiyu
    Resolution: Fixed

Merged in master via 73a547fe46449d11eb078f6a58b44483d40bf5fe.

> TableIdRouter resolves wrong sink table when source regex has multi-digit suffix and sink-table has no back-reference
> ---------------------------------------------------------------------------------------------------------------------
>
>                 Key: FLINK-40148
>                 URL: https://issues.apache.org/jira/browse/FLINK-40148
>             Project: Flink
>          Issue Type: Bug
>          Components: Flink CDC
>    Affects Versions: cdc-3.6.0
>         Environment: flink 1.20.3
> flinkcdc 3.6.0
>            Reporter: peiyu
>            Assignee: peiyu
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: cdc-3.7.0
>
>
> ## Problem
> TableIdRouter.resolveReplacement can produce incorrect sink table names when the source-table
> regex contains a capturing group whose value may span multiple characters (e.g. multi-digit
> suffixes like `13`, `14`, `16`) and the sink-table does not use a `$N` back-reference.
> In this case the unmatched characters after the captured prefix are appended to the configured
> sink-table, producing wrong sink names. For example, given:
>     route:
>       - source-table: db_6.table_([1-9]|1[0-6])
>         sink-table:   new_db_6.table_merged
> `db_6.table_13` is wrongly routed to `new_db_6.table_merged3` (instead of the configured
> `new_db_6.table_merged`), `db_6.table_14` to `new_db_6.table_merged4`, `db_6.table_16` to
> `new_db_6.table_merged6`, etc. Tables 1-9 are not affected because the regex consumes the
> entire single-digit suffix.
> ## Reproduction
> 1. Configure a pipeline with the route above.
> 2. Run the job against a source that has tables `db_6.table_1` … `db_6.table_16`.
> 3. Observe in the logs / sink database that tables 10-16 are written to sink tables named
>    `new_db_6.table_merged0`, `new_db_6.table_merged1`, `new_db_6.table_merged3`, … instead of
>    the single configured sink table.
> A unit test that reproduces the bug (without the fix) is added in this ticket; it fails before
> the patch and passes after.
> ## Root Cause
> `TableIdRouter.resolveReplacement` builds a `Matcher` against the source table id and calls
> `Matcher.find()` followed by `Matcher.replaceAll(sinkTable)`:
>     Matcher matcher = route.f0.matcher(originalTable.toString());
>     if (matcher.find()) {
>         return TableId.parse(matcher.replaceAll(route.f1));
>     }
> Two issues combine to produce the bug:
> 1. `find()` only matches a *prefix* of the source table id. With the regex
>    `([1-9]|1[0-6])`, the alternation `[1-9]` wins first, matching only `1` from input `13`.
> 2. `Matcher.replaceAll` internally calls `reset()` and iterates with `find()`, so it only
>    replaces the prefix matched in step 1 and leaves the trailing characters of the source
>    table id (`0`, `3`, `4`, `5`, `6`) appended to the sink-table. When the sink-table has no
>    `$N` reference, those trailing characters end up in the result.
> The same root cause also makes `find()`-based capturing groups return only the first matched
> digit when the sink-table *does* use `$1`, so the bug is not strictly limited to back-reference
> free sinks.
> ## Proposed Fix
> Switch to `Matcher.matches()` to ensure the entire source table id is consumed, and use
> `Matcher.appendReplacement` / `Matcher.appendTail` to perform the `$N` substitution on the
> matched region only — this avoids the trailing-characters-left-over problem of `replaceAll`.
> ## Tests
> - Add `TableIdRouterTest#testRouteWithoutBackReferenceWithMultiDigitSuffix` to lock down the
>   reported scenario (sink-table has no back-reference).
> - Add `TableIdRouterTest#testRouteWithBackReferenceAndMultiDigitCapture` to cover the related
>   case where sink-table uses `$1` and the captured group must be the full multi-digit value.
> - All existing tests in `TableIdRouterTest`, `TableIdRouterMatchModeTest`, and
>   `SchemaDerivatorTest` continue to pass (27 tests total).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)