You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by "MOBIN (Jira)" <ji...@apache.org> on 2025/02/19 06:39:00 UTC

[jira] [Created] (FLINK-37348) [pipeline-paimon]fix paimon ArrayIndexOutOfBoundsException when add column before first_column

MOBIN created FLINK-37348:
-----------------------------

             Summary: [pipeline-paimon]fix paimon ArrayIndexOutOfBoundsException when add column before first_column
                 Key: FLINK-37348
                 URL: https://issues.apache.org/jira/browse/FLINK-37348
             Project: Flink
          Issue Type: Bug
          Components: Flink CDC
    Affects Versions: cdc-3.3.0
            Reporter: MOBIN


Repeat step:
 # ALTER TABLE table_name add  COLUMN new_first_column bigint first
 # throw ArrayIndexOutOfBoundsException

log:
{code:java}
2025-02-19 11:32:16,586 INFO  org.apache.flink.cdc.runtime.operators.schema.regular.SchemaCoordinator [] - All sink subtask have flushed for table dw_app.cdc_sink19. Start to apply schema change request: 
    SchemaChangeRequest{tableId=dw_app.cdc_sink19, schemaChangeEvent=AddColumnEvent{tableId=dw_app.cdc_sink19, addedColumns=[ColumnWithPosition{column=`new_first_column` BIGINT, position=BEFORE, existedColumnName=first_column}]}, subTaskId=0}
that extracts to:
    AddColumnEvent{tableId=rt_ods.cdc_sink19_add_column_EVOLVE4, addedColumns=[ColumnWithPosition{column=`new_first_column` BIGINT, position=BEFORE, existedColumnName=first_column}]}
2025-02-19 11:32:16,716 ERROR org.apache.flink.cdc.runtime.operators.schema.common.SchemaRegistry [] - An exception was triggered from Schema change applying task. Job will fail now.
org.apache.flink.util.FlinkRuntimeException: Failed to apply schema change event.
    at org.apache.flink.cdc.runtime.operators.schema.regular.SchemaCoordinator.lambda$startSchemaChangesEvolve$0(SchemaCoordinator.java:290) ~[flink-cdc-dist-3.3.0.jar:3.3.0]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_65]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_65]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_65]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_65]
    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_65]
Caused by: java.lang.ArrayIndexOutOfBoundsException: -1
    at java.util.ArrayList.elementData(ArrayList.java:418) ~[?:1.8.0_65]
    at java.util.ArrayList.get(ArrayList.java:431) ~[?:1.8.0_65]
    at org.apache.flink.cdc.connectors.paimon.sink.PaimonMetadataApplier.applyAddColumnWithBeforePosition(PaimonMetadataApplier.java:275) ~[flink-cdc-pipeline-connector-paimon-3.3.0.jar:3.3.0]
    at org.apache.flink.cdc.connectors.paimon.sink.PaimonMetadataApplier.applyAddColumnEventWithPosition(PaimonMetadataApplier.java:237) ~[flink-cdc-pipeline-connector-paimon-3.3.0.jar:3.3.0]
    at org.apache.flink.cdc.connectors.paimon.sink.PaimonMetadataApplier.applyAddColumn(PaimonMetadataApplier.java:206) ~[flink-cdc-pipeline-connector-paimon-3.3.0.jar:3.3.0]
    at org.apache.flink.cdc.connectors.paimon.sink.PaimonMetadataApplier.lambda$applySchemaChange$0(PaimonMetadataApplier.java:127) ~[flink-cdc-pipeline-connector-paimon-3.3.0.jar:3.3.0]
    at org.apache.flink.cdc.common.event.visitor.SchemaChangeEventVisitor.visit(SchemaChangeEventVisitor.java:47) ~[flink-cdc-dist-3.3.0.jar:3.3.0]
    at org.apache.flink.cdc.connectors.paimon.sink.PaimonMetadataApplier.applySchemaChange(PaimonMetadataApplier.java:124) ~[flink-cdc-pipeline-connector-paimon-3.3.0.jar:3.3.0]
    at org.apache.flink.cdc.runtime.operators.schema.regular.SchemaCoordinator.applyAndUpdateEvolvedSchemaChange(SchemaCoordinator.java:434) ~[flink-cdc-dist-3.3.0.jar:3.3.0]
    at org.apache.flink.cdc.runtime.operators.schema.regular.SchemaCoordinator.applySchemaChange(SchemaCoordinator.java:401) ~[flink-cdc-dist-3.3.0.jar:3.3.0]
    at org.apache.flink.cdc.runtime.operators.schema.regular.SchemaCoordinator.lambda$startSchemaChangesEvolve$0(SchemaCoordinator.java:288) ~[flink-cdc-dist-3.3.0.jar:3.3.0]
    ... 5 more {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)