You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "0dunay0 (via GitHub)" <gi...@apache.org> on 2025/10/01 13:28:50 UTC
[PR] [paimon-flink-cdc] Add skip-corrupt-record configuration for Kafka CDC with PII-safe logging [paimon]
0dunay0 opened a new pull request, #6360:
URL: https://github.com/apache/paimon/pull/6360
### Purpose
Fixes job failures when Kafka CDC ingestion encounters corrupt or unparsable records. Instead of crashing the job, users can now configure the parser to skip invalid records with PII-safe logging.
This feature is already available for [MySQL sources](https://github.com/apache/paimon/blob/master/paimon-flink/paimon-flink-cdc/src/main/java/org/apache/paimon/flink/sink/cdc/CdcRecordStoreWriteOperator.java#L64-L75).
<!-- What is the purpose of the change -->
### Tests
It's a difficult thing to test to produce invalid avro messages to a topic but it's been tested with a live job in our cluster.
`2025-10-01 00:04:51,433 [] WARN org.apache.paimon.flink.action.cdc.format.AbstractRecordParser [] - Skipping corrupt or unparsable source record.`
### Documentation
Docs has been updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [paimon-flink-cdc] Add skip-corrupt-record configuration for Kafka CDC with PII-safe logging [paimon]
Posted by "0dunay0 (via GitHub)" <gi...@apache.org>.
0dunay0 closed pull request #6360: [paimon-flink-cdc] Add skip-corrupt-record configuration for Kafka CDC with PII-safe logging
URL: https://github.com/apache/paimon/pull/6360
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org