You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "MOBIN (Jira)" <ji...@apache.org> on 2024/12/06 07:51:00 UTC
[jira] [Commented] (FLINK-36858) [pipeline-kafka] JsonRowDataSerializationSchema compatibility bug
[ https://issues.apache.org/jira/browse/FLINK-36858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17903568#comment-17903568 ]
MOBIN commented on FLINK-36858:
-------------------------------
It seems that flink cdc has not yet fully adapted to flink 1.20, but [e2e test |https://github.com/apache/flink-cdc/blob/ee9cd828c3ba822570cb349f7ce978234086aeb6/flink-cdc-e2e-tests/flink-cdc-pipeline-e2e-tests/src/test/java/org/apache/flink/cdc/pipeline/tests/utils/PipelineTestEnvironment.java#L98]has tests for flink 1.20
> [pipeline-kafka] JsonRowDataSerializationSchema compatibility bug
> -----------------------------------------------------------------
>
> Key: FLINK-36858
> URL: https://issues.apache.org/jira/browse/FLINK-36858
> Project: Flink
> Issue Type: Bug
> Components: Flink CDC
> Affects Versions: cdc-3.2.1
> Reporter: MOBIN
> Priority: Major
> Attachments: image-2024-12-06-15-20-02-727.png
>
>
> minimum repetition step：
> Flink 1.20
> Flink cdc master
> {code:java}
> source:
> type: mysql
> name: MySQL Source
> hostname: localhost
> port: 3306
> username: test
> password: test
> tables: testDB.testTable
> server-id: 5206
> server-time-zone: Asia/Shanghai
> schema-change.enabled: true
>
> sink:
> type: kafka
> name: Kafka Sink
> properties.bootstrap.servers: localhost:9092
> topic: test_topic
> pipeline:
> name: MySQL to Kafka Pipeline
> parallelism: 1 {code}
> error message:
> {code:java}
> Caused by: java.lang.NoSuchMethodError: 'void org.apache.flink.formats.json.JsonRowDataSerializationSchema.<init>(org.apache.flink.table.types.logical.RowType, org.apache.flink.formats.common.TimestampFormat, org.apache.flink.formats.json.JsonFormatOptions$MapNullKeyMode, java.lang.String, boolean)'
> at org.apache.flink.cdc.connectors.kafka.serialization.JsonSerializationSchema.buildSerializationForPrimaryKey(JsonSerializationSchema.java:134)
> at org.apache.flink.cdc.connectors.kafka.serialization.JsonSerializationSchema.serialize(JsonSerializationSchema.java:101)
> at org.apache.flink.cdc.connectors.kafka.serialization.JsonSerializationSchema.serialize(JsonSerializationSchema.java:47)
> at org.apache.flink.cdc.connectors.kafka.sink.PipelineKafkaRecordSerializationSchema.serialize(PipelineKafkaRecordSerializationSchema.java:99)
> at org.apache.flink.cdc.connectors.kafka.sink.PipelineKafkaRecordSerializationSchema.serialize(PipelineKafkaRecordSerializationSchema.java:44) {code}
> Cause of error：
> The JsonRowDataSerializationSchema constructor of Flink 1.20 has an additional ignoreNullFields parameter, which causes this [part of the code|https://github.com/apache/flink-cdc/blob/ee9cd828c3ba822570cb349f7ce978234086aeb6/flink-cdc-connect/flink-cdc-pipeline-connectors/flink-cdc-pipeline-connector-kafka/src/main/java/org/apache/flink/cdc/connectors/kafka/serialization/JsonSerializationSchema.java#L134] to throw a NoSuchMethodError
> !image-2024-12-06-15-20-02-727.png|width=1315,height=364!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)