You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "tstangenberg (via GitHub)" <gi...@apache.org> on 2026/03/02 14:37:33 UTC

[I] [Bug]: KafkaConnectUtils.debeziumRecordInstant() fails with NullPointerException on DELETE operations [beam]

tstangenberg opened a new issue, #37738:
URL: https://github.com/apache/beam/issues/37738

   ### What happened?
   
   The `debeziumRecordInstant()` method in KafkaConnectUtils throws a NullPointerException when processing Debezium DELETE events because it doesn't handle the case where valueSchema is null.
   
   **Steps to Reproduce**
   
   1. Set up a Debezium pipeline using DebeziumIO.read()
   2. Perform a DELETE operation on a record in the source database
   3. The pipeline crashes with a NullPointerException
   
   **Expected Behavior**
   
   The method should handle DELETE operations gracefully by extracting the timestamp from the sourceOffset metadata, specifically from the ts_usec field.
   
   **Actual Behavior**
   
   ```
   NullPointerException: Cannot invoke "org.apache.kafka.connect.data.Schema.type()" because the return value
           of "org.apache.kafka.connect.source.SourceRecord.valueSchema()" is null
   	at org.apache.beam.io.debezium.KafkaConnectUtils.debeziumRecordInstant(KafkaConnectUtils.java:83)
   ```
   
   **Root Cause**
   For DELETE operations, Debezium sets and to null, but the timestamp information is available in `sourceOffset.ts_usec`: `value``valueSchema`
   
   ```
   SourceRecord{
     sourcePartition={server=beam-debezium-connector}, 
     sourceOffset={lsn_proc=42600784, messageType=DELETE, lsn_commit=42600728, lsn=42600784, txId=1073, ts_usec=1772459804615258}
     ConnectRecord{..., value=null, valueSchema=null, ...}
   }
   ```
   
   ### Issue Priority
   
   Priority: 2 (default / most bugs should be filed as P2)
   
   ### Issue Components
   
   - [ ] Component: Python SDK
   - [x] Component: Java SDK
   - [ ] Component: Go SDK
   - [ ] Component: Typescript SDK
   - [ ] Component: IO connector
   - [ ] Component: Beam YAML
   - [ ] Component: Beam examples
   - [ ] Component: Beam playground
   - [ ] Component: Beam katas
   - [ ] Component: Website
   - [ ] Component: Infrastructure
   - [ ] Component: Spark Runner
   - [ ] Component: Flink Runner
   - [ ] Component: Samza Runner
   - [ ] Component: Twister2 Runner
   - [ ] Component: Hazelcast Jet Runner
   - [ ] Component: Google Cloud Dataflow Runner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug]: KafkaConnectUtils.debeziumRecordInstant() fails with NullPointerException on DELETE operations [beam]

Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad closed issue #37738: [Bug]: KafkaConnectUtils.debeziumRecordInstant() fails with NullPointerException on DELETE operations
URL: https://github.com/apache/beam/issues/37738


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug]: KafkaConnectUtils.debeziumRecordInstant() fails with NullPointerException on DELETE operations [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #37738:
URL: https://github.com/apache/beam/issues/37738#issuecomment-4024909866

   The problem is that Debezium issues two SourceRecords on a deletion event. It is the tombstone message causing the problem here. For now, one can set DebeziumIO.Read configuration `.withConnectionProperty("tombstones.on.delete", "false")` as a workaround.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org