You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Mickael Maison (Jira)" <ji...@apache.org> on 2023/05/31 13:56:00 UTC
[jira] [Resolved] (KAFKA-8713) [Connect] JsonConverter NULL Values are replaced by default values even in NULLABLE fields
[ https://issues.apache.org/jira/browse/KAFKA-8713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mickael Maison resolved KAFKA-8713.
-----------------------------------
Resolution: Fixed
> [Connect] JsonConverter NULL Values are replaced by default values even in NULLABLE fields
> ------------------------------------------------------------------------------------------
>
> Key: KAFKA-8713
> URL: https://issues.apache.org/jira/browse/KAFKA-8713
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Affects Versions: 2.0.0, 2.0.1, 2.1.0, 2.2.0, 2.3.0, 2.2.1
> Reporter: Cheng Pan
> Assignee: Mickael Maison
> Priority: Major
> Labels: needs-kip
> Fix For: 3.5.0
>
>
> Class JsonConverter line: 582
> {code:java}
> private static JsonNode convertToJson(Schema schema, Object logicalValue) {
> if (logicalValue == null) {
> if (schema == null) // Any schema is valid and we don't have a default, so treat this as an optional schema
> return null;
> if (schema.defaultValue() != null)
> return convertToJson(schema, schema.defaultValue());
> if (schema.isOptional())
> return JsonNodeFactory.instance.nullNode();
> throw new DataException("Conversion error: null value for field that is required and has no default value");
> }
> ....
> }
> {code}
> h1.Expect:
> Value `null` is valid for an optional filed, even though the filed has a default value.
> Only when field is required, the converter return default value fallback when value is `null`.
> h1.Actual:
> Always return default value if `null` was given.
> h1. Example:
> I'm not sure if the current behavior is the exactly expected, but at least on MySQL, a table define as
> {code:sql}
> create table t1 {
> name varchar(40) not null,
> create_time datetime default '1999-01-01 11:11:11' null,
> update_time datetime default '1999-01-01 11:11:11' null
> }
> {code}
> Just insert a record:
> {code:sql}
> INSERT INTO `t1` (`name`, `update_time`) VALUES ('kafka', null);
> {code}
> The result is:
> {code:json}
> {
> "name": "kafka",
> "create_time": "1999-01-01 11:11:11",
> "update_time": null
> }
> {code}
> But when I use debezium pull binlog and send the record to Kafka with JsonConverter, the result changed to:
> {code:json}
> {
> "name": "kafka",
> "create_time": "1999-01-01 11:11:11",
> "update_time": "1999-01-01 11:11:11"
> }
> {code}
> For more details, see: https://issues.jboss.org/browse/DBZ-1064
--
This message was sent by Atlassian Jira
(v8.20.10#820010)