You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Hisoka-X (via GitHub)" <gi...@apache.org> on 2026/02/05 14:18:32 UTC
[PR] [FLINK-37837][core] Always add create.table to include.schema.changes option [flink-cdc]
Hisoka-X opened a new pull request, #4262:
URL: https://github.com/apache/flink-cdc/pull/4262
This close https://issues.apache.org/jira/browse/FLINK-37837
This pull request ensures that the `CreateTableEvent` is always included in the set of schema change events unless the user explicitly excludes it. This change guarantees the foundational event is present for proper schema evolution handling.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37837][core] Always add create.table to include.schema.changes option [flink-cdc]
Posted by "Hisoka-X (via GitHub)" <gi...@apache.org>.
Hisoka-X commented on PR #4262:
URL: https://github.com/apache/flink-cdc/pull/4262#issuecomment-3976153979
> I think that` always add create.table to include.schema.changes` option is not a good idea ,because user can create table without [flink-cdc](https://issues.apache.org/jira/browse/FLINK-cdc) and others
Users can set create.table into exclude.schema.changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37837][core] Always add create.table to include.schema.changes option [flink-cdc]
Posted by "Hisoka-X (via GitHub)" <gi...@apache.org>.
Hisoka-X closed pull request #4262: [FLINK-37837][core] Always add create.table to include.schema.changes option
URL: https://github.com/apache/flink-cdc/pull/4262
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37837][core] Always add create.table to include.schema.changes option [flink-cdc]
Posted by "Hisoka-X (via GitHub)" <gi...@apache.org>.
Hisoka-X commented on code in PR #4262:
URL: https://github.com/apache/flink-cdc/pull/4262#discussion_r2959929317
##########
flink-cdc-cli/src/main/java/org/apache/flink/cdc/cli/parser/YamlPipelineDefinitionParser.java:
##########
@@ -226,6 +227,19 @@ private SinkDef toSinkDef(JsonNode sinkNode, SchemaChangeBehavior schemaChangeBe
Arrays.stream(SchemaChangeEventTypeFamily.ALL)
.map(SchemaChangeEventType::getTag)
.forEach(includedSETypes::add);
+ } else {
+ // CreateTableEvent is always required as the foundation for all subsequent processing.
+ // Automatically add it if not explicitly excluded by user.
+ // Use resolveSchemaEvolutionTag to properly handle both exact tags and family tags.
+ boolean createTableExplicitlyExcluded =
+ excludedSETypes.stream()
+ .flatMap(
+ tag -> ChangeEventUtils.resolveSchemaEvolutionTag(tag).stream())
+ .anyMatch(type -> type == SchemaChangeEventType.CREATE_TABLE);
+ if (!createTableExplicitlyExcluded
+ && !includedSETypes.contains(SchemaChangeEventType.CREATE_TABLE.getTag())) {
+ includedSETypes.add(SchemaChangeEventType.CREATE_TABLE.getTag());
Review Comment:
done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37837][core] Always add create.table to include.schema.changes option [flink-cdc]
Posted by "lvyanquan (via GitHub)" <gi...@apache.org>.
lvyanquan merged PR #4262:
URL: https://github.com/apache/flink-cdc/pull/4262
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37837][core] Always add create.table to include.schema.changes option [flink-cdc]
Posted by "Hisoka-X (via GitHub)" <gi...@apache.org>.
Hisoka-X commented on PR #4262:
URL: https://github.com/apache/flink-cdc/pull/4262#issuecomment-4009115615
cc @lvyanquan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37837][core] Always add create.table to include.schema.changes option [flink-cdc]
Posted by "ruanhang1993 (via GitHub)" <gi...@apache.org>.
ruanhang1993 commented on code in PR #4262:
URL: https://github.com/apache/flink-cdc/pull/4262#discussion_r2958439141
##########
flink-cdc-cli/src/main/java/org/apache/flink/cdc/cli/parser/YamlPipelineDefinitionParser.java:
##########
@@ -226,6 +227,19 @@ private SinkDef toSinkDef(JsonNode sinkNode, SchemaChangeBehavior schemaChangeBe
Arrays.stream(SchemaChangeEventTypeFamily.ALL)
.map(SchemaChangeEventType::getTag)
.forEach(includedSETypes::add);
+ } else {
+ // CreateTableEvent is always required as the foundation for all subsequent processing.
+ // Automatically add it if not explicitly excluded by user.
+ // Use resolveSchemaEvolutionTag to properly handle both exact tags and family tags.
+ boolean createTableExplicitlyExcluded =
+ excludedSETypes.stream()
+ .flatMap(
+ tag -> ChangeEventUtils.resolveSchemaEvolutionTag(tag).stream())
+ .anyMatch(type -> type == SchemaChangeEventType.CREATE_TABLE);
+ if (!createTableExplicitlyExcluded
+ && !includedSETypes.contains(SchemaChangeEventType.CREATE_TABLE.getTag())) {
+ includedSETypes.add(SchemaChangeEventType.CREATE_TABLE.getTag());
Review Comment:
@Hisoka-X Please update the docs about this behavior changing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-37837][core] Always add create.table to include.schema.changes option [flink-cdc]
Posted by "ThorneANN (via GitHub)" <gi...@apache.org>.
ThorneANN commented on PR #4262:
URL: https://github.com/apache/flink-cdc/pull/4262#issuecomment-3976763278
and i think the jira's error was
> > I think that` always add create.table to include.schema.changes` option is not a good idea ,because user can create table without [flink-cdc](https://issues.apache.org/jira/browse/FLINK-cdc) and others
>
> Users can set create.table into exclude.schema.changes.
I think this is the correct processing logic, if users always create tables first, and creating tables may be a failure that leads to task failure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org