You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "linjianchang (via GitHub)" <gi...@apache.org> on 2025/01/03 10:07:44 UTC
[PR] [FLINK-36999][cdc-source-connectors] When the source field type is de… [flink-cdc]
linjianchang opened a new pull request, #3833:
URL: https://github.com/apache/flink-cdc/pull/3833
When the source field type is decimal and the value is null, a null pointer exception occurs during deserialization.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-36999][cdc-source-connectors] When the source type is oracle and field type is nember and the value is null, a null pointer exception occurs during deserialization. [flink-cdc]
Posted by "linjianchang (via GitHub)" <gi...@apache.org>.
linjianchang commented on code in PR #3833:
URL: https://github.com/apache/flink-cdc/pull/3833#discussion_r2055105276
##########
flink-cdc-connect/flink-cdc-source-connectors/flink-connector-debezium/src/main/java/org/apache/flink/cdc/debezium/event/DebeziumSchemaDataTypeInference.java:
##########
@@ -188,6 +188,10 @@ protected DataType inferBytes(Object value, Schema schema) {
protected DataType inferStruct(Object value, Schema schema) {
Struct struct = (Struct) value;
if (VariableScaleDecimal.LOGICAL_NAME.equals(schema.name())) {
+ if (struct == null) {
+ // set the default value
+ return DataTypes.DECIMAL(10, 0);
Review Comment:
> @linjianchang Can you add unit test for this scenario？
have added unit test,please review again,thanks@GOODBOY008
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-36999][cdc-source-connectors] When the source type is oracle and field type is nember and the value is null, a null pointer exception occurs during deserialization. [flink-cdc]
Posted by "linjianchang (via GitHub)" <gi...@apache.org>.
linjianchang commented on code in PR #3833:
URL: https://github.com/apache/flink-cdc/pull/3833#discussion_r2055105596
##########
flink-cdc-connect/flink-cdc-source-connectors/flink-connector-debezium/src/main/java/org/apache/flink/cdc/debezium/event/DebeziumSchemaDataTypeInference.java:
##########
@@ -188,6 +188,10 @@ protected DataType inferBytes(Object value, Schema schema) {
protected DataType inferStruct(Object value, Schema schema) {
Struct struct = (Struct) value;
if (VariableScaleDecimal.LOGICAL_NAME.equals(schema.name())) {
+ if (struct == null) {
+ // set the default value
+ return DataTypes.DECIMAL(10, 0);
Review Comment:
@GOODBOY008 have added unit test,please review again,thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-36999][cdc-source-connectors] When the source type is oracle and field type is nember and the value is null, a null pointer exception occurs during deserialization. [flink-cdc]
Posted by "linjianchang (via GitHub)" <gi...@apache.org>.
linjianchang commented on code in PR #3833:
URL: https://github.com/apache/flink-cdc/pull/3833#discussion_r2053571487
##########
flink-cdc-connect/flink-cdc-source-connectors/flink-connector-debezium/src/main/java/org/apache/flink/cdc/debezium/event/DebeziumSchemaDataTypeInference.java:
##########
@@ -188,6 +188,10 @@ protected DataType inferBytes(Object value, Schema schema) {
protected DataType inferStruct(Object value, Schema schema) {
Struct struct = (Struct) value;
if (VariableScaleDecimal.LOGICAL_NAME.equals(schema.name())) {
+ if (struct == null) {
+ // set the default value
+ return DataTypes.DECIMAL(10, 0);
Review Comment:
> @linjianchang Can you add unit test for this scenario？
have add unit test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-36999][cdc-source-connectors] When the source type is oracle and field type is nember and the value is null, a null pointer exception occurs during deserialization. [flink-cdc]
Posted by "linjianchang (via GitHub)" <gi...@apache.org>.
linjianchang commented on code in PR #3833:
URL: https://github.com/apache/flink-cdc/pull/3833#discussion_r2055105276
##########
flink-cdc-connect/flink-cdc-source-connectors/flink-connector-debezium/src/main/java/org/apache/flink/cdc/debezium/event/DebeziumSchemaDataTypeInference.java:
##########
@@ -188,6 +188,10 @@ protected DataType inferBytes(Object value, Schema schema) {
protected DataType inferStruct(Object value, Schema schema) {
Struct struct = (Struct) value;
if (VariableScaleDecimal.LOGICAL_NAME.equals(schema.name())) {
+ if (struct == null) {
+ // set the default value
+ return DataTypes.DECIMAL(10, 0);
Review Comment:
> @linjianchang Can you add unit test for this scenario？
have added unit test,please review again,thanks@GOODBOY008
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-36999][cdc-source-connectors] When the source field type is de… [flink-cdc]
Posted by "linjianchang (via GitHub)" <gi...@apache.org>.
linjianchang commented on PR #3833:
URL: https://github.com/apache/flink-cdc/pull/3833#issuecomment-2572228210
**source oracle table structure：**
create table PORTAL.SYSTEM_USER
(
sys_user_id NUMBER not null,
staff_id NUMBER not null,
sys_user_code VARCHAR2(250),
password VARCHAR2(250),
pwd_err_cnt NUMBER not null,
pwd_sms_tel NUMBER
)
When the pwd_sms_tel field value of the nember type is null, a null pointer exception will be reported.
**Exception:**
`java.lang.NullPointerException
at io.debezium.data.VariableScaleDecimal.toLogical(VariableScaleDecimal.java:103)
at com.ververica.cdc.debezium.event.DebeziumSchemaDataTypeInference.inferStruct(DebeziumSchemaDataTypeInference.java:190)
at com.ververica.cdc.connectors.oracle.source.OracleSchemaDataTypeInference.inferStruct(OracleSchemaDataTypeInference.java:40)
at com.ververica.cdc.debezium.event.DebeziumSchemaDataTypeInference.infer(DebeziumSchemaDataTypeInference.java:81)
at com.ververica.cdc.debezium.event.DebeziumSchemaDataTypeInference.infer(DebeziumSchemaDataTypeInference.java:56)
at com.ververica.cdc.debezium.event.DebeziumSchemaDataTypeInference.lambda$inferStruct$0(DebeziumSchemaDataTypeInference.java:199)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:546)
at java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
at java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:505)
at com.ververica.cdc.debezium.event.DebeziumSchemaDataTypeInference.inferStruct(DebeziumSchemaDataTypeInference.java:200)
at com.ververica.cdc.connectors.oracle.source.OracleSchemaDataTypeInference.inferStruct(OracleSchemaDataTypeInference.java:40)
at com.ververica.cdc.debezium.event.DebeziumSchemaDataTypeInference.infer(DebeziumSchemaDataTypeInference.java:81)
at com.ververica.cdc.debezium.event.DebeziumSchemaDataTypeInference.infer(DebeziumSchemaDataTypeInference.java:56)
at com.ververica.cdc.debezium.event.DebeziumEventDeserializationSchema.extractDataRecord(DebeziumEventDeserializationSchema.java:157)
at com.ververica.cdc.debezium.event.DebeziumEventDeserializationSchema.extractAfterDataRecord(DebeziumEventDeserializationSchema.java:153)
at com.ververica.cdc.debezium.event.DebeziumEventDeserializationSchema.deserializeDataChangeRecord(DebeziumEventDeserializationSchema.java:119)
at com.ververica.cdc.debezium.event.SourceRecordEventDeserializer.deserialize(SourceRecordEventDeserializer.java:47)
at com.ververica.cdc.debezium.event.DebeziumEventDeserializationSchema.deserialize(DebeziumEventDeserializationSchema.java:106)
at com.ververica.cdc.connectors.base.source.reader.IncrementalSourceRecordEmitter.emitElement(IncrementalSourceRecordEmitter.java:172)
at com.ververica.cdc.connectors.base.source.reader.IncrementalSourceRecordEmitter.processElement(IncrementalSourceRecordEmitter.java:134)
at com.ververica.cdc.connectors.oracle.source.reader.OraclePipelineRecordEmitter.processElement(OraclePipelineRecordEmitter.java:90)
at com.ververica.cdc.connectors.base.source.reader.IncrementalSourceRecordEmitter.emitRecord(IncrementalSourceRecordEmitter.java:98)
at com.ververica.cdc.connectors.base.source.reader.IncrementalSourceRecordEmitter.emitRecord(IncrementalSourceRecordEmitter.java:59)
at org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:143)
at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:389)
at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:68)
at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:545)
at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)
at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:836)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:785)
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935)
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550)
at java.lang.Thread.run(Thread.java:750)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-36999][cdc-source-connectors] When the source type is oracle and field type is nember and the value is null, a null pointer exception occurs during deserialization. [flink-cdc]
Posted by "linjianchang (via GitHub)" <gi...@apache.org>.
linjianchang closed pull request #3833: [FLINK-36999][cdc-source-connectors] When the source type is oracle and field type is nember and the value is null, a null pointer exception occurs during deserialization.
URL: https://github.com/apache/flink-cdc/pull/3833
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-36999][cdc-source-connectors] When the source type is oracle and field type is nember and the value is null, a null pointer exception occurs during deserialization. [flink-cdc]
Posted by "linjianchang (via GitHub)" <gi...@apache.org>.
linjianchang commented on code in PR #3833:
URL: https://github.com/apache/flink-cdc/pull/3833#discussion_r2053571487
##########
flink-cdc-connect/flink-cdc-source-connectors/flink-connector-debezium/src/main/java/org/apache/flink/cdc/debezium/event/DebeziumSchemaDataTypeInference.java:
##########
@@ -188,6 +188,10 @@ protected DataType inferBytes(Object value, Schema schema) {
protected DataType inferStruct(Object value, Schema schema) {
Struct struct = (Struct) value;
if (VariableScaleDecimal.LOGICAL_NAME.equals(schema.name())) {
+ if (struct == null) {
+ // set the default value
+ return DataTypes.DECIMAL(10, 0);
Review Comment:
> @linjianchang Can you add unit test for this scenario？
have add unit test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-36999][cdc-source-connectors] When the source type is oracle and field type is nember and the value is null, a null pointer exception occurs during deserialization. [flink-cdc]
Posted by "linjianchang (via GitHub)" <gi...@apache.org>.
linjianchang closed pull request #3833: [FLINK-36999][cdc-source-connectors] When the source type is oracle and field type is nember and the value is null, a null pointer exception occurs during deserialization.
URL: https://github.com/apache/flink-cdc/pull/3833
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-36999][cdc-source-connectors] When the source type is oracle and field type is nember and the value is null, a null pointer exception occurs during deserialization. [flink-cdc]
Posted by "GOODBOY008 (via GitHub)" <gi...@apache.org>.
GOODBOY008 commented on code in PR #3833:
URL: https://github.com/apache/flink-cdc/pull/3833#discussion_r1915880779
##########
flink-cdc-connect/flink-cdc-source-connectors/flink-connector-debezium/src/main/java/org/apache/flink/cdc/debezium/event/DebeziumSchemaDataTypeInference.java:
##########
@@ -188,6 +188,10 @@ protected DataType inferBytes(Object value, Schema schema) {
protected DataType inferStruct(Object value, Schema schema) {
Struct struct = (Struct) value;
if (VariableScaleDecimal.LOGICAL_NAME.equals(schema.name())) {
+ if (struct == null) {
+ // set the default value
+ return DataTypes.DECIMAL(10, 0);
Review Comment:
```suggestion
return DataTypes.DECIMAL(DecimalType.DEFAULT_PRECISION, DecimalType.DEFAULT_SCALE);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-36999][cdc-source-connectors] When the source type is oracle and field type is nember and the value is null, a null pointer exception occurs during deserialization. [flink-cdc]
Posted by "GOODBOY008 (via GitHub)" <gi...@apache.org>.
GOODBOY008 commented on code in PR #3833:
URL: https://github.com/apache/flink-cdc/pull/3833#discussion_r1915881357
##########
flink-cdc-connect/flink-cdc-source-connectors/flink-connector-debezium/src/main/java/org/apache/flink/cdc/debezium/event/DebeziumSchemaDataTypeInference.java:
##########
@@ -188,6 +188,10 @@ protected DataType inferBytes(Object value, Schema schema) {
protected DataType inferStruct(Object value, Schema schema) {
Struct struct = (Struct) value;
if (VariableScaleDecimal.LOGICAL_NAME.equals(schema.name())) {
+ if (struct == null) {
+ // set the default value
+ return DataTypes.DECIMAL(10, 0);
Review Comment:
@linjianchang Can you add unit test for this scenario？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org