You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "zhongyuandong (Jira)" <ji...@apache.org> on 2024/08/23 03:09:00 UTC

[jira] [Updated] (FLINK-36141) Set the scanNewlyAddedTableEnabled is true, but does not support automatic identification new table

     [ https://issues.apache.org/jira/browse/FLINK-36141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

zhongyuandong updated FLINK-36141:
----------------------------------
    Description: 
1、 see the flink-cdc3.0.1 debezium reader stage, specially added scanNewlyAddedTableEnabled is true filtered out, so lead to be inaccessible to the new table without restart? Based on 3.0.1 version design, testing the scanNewlyAddedTableEnabled = false (lost manually add table, restart operations cannot identify to), but it can automatically recognize new tables with regular expressions without restarting them, is this a bug or purposely so design? (Why is it designed this way?) 
2、 flink-cdc2.3.0 without this logic, can support the identification of the new table. TODO: there is still very little chance that we can't capture new added table. Is this the reason why flink-cdc2.4.0 and later versions no longer support dynamic recognition of new tables?
3. Which version solves this problem? How to solve it?

!image-2024-08-23-11-08-31-324.png!

!image-2024-08-23-11-08-44-156.png!

  was:
1、 see the flink-cdc3.0.1 debezium reader stage, specially added scanNewlyAddedTableEnabled is true filtered out, so lead to be inaccessible to the new table without restart? Based on 3.0.1 version design, testing the scanNewlyAddedTableEnabled = false (lost manually add table, restart operations cannot identify to), but it can automatically recognize new tables with regular expressions without restarting them, is this a bug or purposely so design? (Why is it designed this way?) 
2、 flink-cdc2.3.0 without this logic, can support the identification of the new table. TODO: there is still very little chance that we can't capture new added table. Is this the reason why flink-cdc2.4.0 and later versions no longer support dynamic recognition of new tables?
3. Which version solves this problem? How to solve it?


> Set the scanNewlyAddedTableEnabled is true, but does not support automatic identification new table
> ---------------------------------------------------------------------------------------------------
>
>                 Key: FLINK-36141
>                 URL: https://issues.apache.org/jira/browse/FLINK-36141
>             Project: Flink
>          Issue Type: Bug
>          Components: Flink CDC
>    Affects Versions: cdc-3.1.0
>         Environment: !image-2024-08-23-11-07-38-182.png!!image-2024-08-23-11-07-26-384.png!
>            Reporter: zhongyuandong
>            Priority: Critical
>         Attachments: image-2024-08-23-11-07-26-384.png, image-2024-08-23-11-07-38-182.png, image-2024-08-23-11-08-31-324.png, image-2024-08-23-11-08-44-156.png
>
>
> 1、 see the flink-cdc3.0.1 debezium reader stage, specially added scanNewlyAddedTableEnabled is true filtered out, so lead to be inaccessible to the new table without restart? Based on 3.0.1 version design, testing the scanNewlyAddedTableEnabled = false (lost manually add table, restart operations cannot identify to), but it can automatically recognize new tables with regular expressions without restarting them, is this a bug or purposely so design? (Why is it designed this way?) 
> 2、 flink-cdc2.3.0 without this logic, can support the identification of the new table. TODO: there is still very little chance that we can't capture new added table. Is this the reason why flink-cdc2.4.0 and later versions no longer support dynamic recognition of new tables?
> 3. Which version solves this problem? How to solve it?
> !image-2024-08-23-11-08-31-324.png!
> !image-2024-08-23-11-08-44-156.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)