You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "yangyu (Jira)" <ji...@apache.org> on 2025/07/23 07:26:00 UTC

[jira] [Commented] (FLINK-38134) Cannot recognize DropColumnEvent in method SchemaMergingUtils.getSchemaDifference.

    [ https://issues.apache.org/jira/browse/FLINK-38134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18009153#comment-18009153 ] 

yangyu commented on FLINK-38134:
--------------------------------

we solve it as:
{code:java}
public static List<SchemaChangeEvent> getSchemaDifference(
TableId tableId, @Nullable Schema beforeSchema, Schema afterSchema) {
if (beforeSchema == null) {
return Collections.singletonList(new CreateTableEvent(tableId, afterSchema));
}

Map<String, Column> beforeColumns =
beforeSchema.getColumns().stream()
.collect(Collectors.toMap(Column::getName, col -> col));

Map<String, DataType> oldTypeMapping = new HashMap<>();
Map<String, DataType> newTypeMapping = new HashMap<>();
List<AddColumnEvent.ColumnWithPosition> appendedColumns = new ArrayList<>();

String afterWhichColumnPosition = null;
for (Column afterColumn : afterSchema.getColumns()) {
String columnName = afterColumn.getName();
DataType afterType = afterColumn.getType();
if (beforeColumns.containsKey(columnName)) {
DataType beforeType = beforeColumns.get(columnName).getType();
if (!Objects.equals(beforeType, afterType)) {
oldTypeMapping.put(columnName, beforeType);
newTypeMapping.put(columnName, afterType);
}
beforeColumns.remove(columnName);
} else {
if (afterWhichColumnPosition == null) {
appendedColumns.add(
new AddColumnEvent.ColumnWithPosition(
afterColumn, AddColumnEvent.ColumnPosition.FIRST, null));
} else {
appendedColumns.add(
new AddColumnEvent.ColumnWithPosition(
afterColumn,
AddColumnEvent.ColumnPosition.AFTER,
afterWhichColumnPosition));
}
}
afterWhichColumnPosition = afterColumn.getName();
}

List<SchemaChangeEvent> schemaChangeEvents = new ArrayList<>();
if (!appendedColumns.isEmpty()) {
schemaChangeEvents.add(new AddColumnEvent(tableId, appendedColumns));
}

if (!newTypeMapping.isEmpty()) {
schemaChangeEvents.add(
new AlterColumnTypeEvent(tableId, newTypeMapping, oldTypeMapping));
}

if (!beforeColumns.isEmpty()) {
schemaChangeEvents.add(
new DropColumnEvent(tableId, new ArrayList<>(beforeColumns.keySet())));
}
return schemaChangeEvents;
}{code}

> Cannot recognize DropColumnEvent in method SchemaMergingUtils.getSchemaDifference.
> ----------------------------------------------------------------------------------
>
>                 Key: FLINK-38134
>                 URL: https://issues.apache.org/jira/browse/FLINK-38134
>             Project: Flink
>          Issue Type: Bug
>          Components: Flink CDC
>            Reporter: yangyu
>            Priority: Major
>
> Only AddColumnEvent and AlterColumnTypeEvent can be recognized now.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)