You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Mahendran Ponnusamy (Jira)" <ji...@apache.org> on 2021/10/12 19:05:00 UTC
[jira] [Comment Edited] (FLINK-24494) Avro Confluent Registry SQL
not fails to write to the topic with schema with default value for int
[ https://issues.apache.org/jira/browse/FLINK-24494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17427871#comment-17427871 ]
Mahendran Ponnusamy edited comment on FLINK-24494 at 10/12/21, 7:04 PM:
------------------------------------------------------------------------
Thank you [~MartijnVisser], I figured the issue is not with the default, it is with the schema name that is generated by the connector.
The subjects has the below schema registered
{quote}{"type":"record","name":"SimpleCustomer","namespace":"com.xxxx.yyyy.zzzz.example.model","fields":[\{"name":"customerId","type":"string"},\{"name":"age","type":"int","default":0}]}{quote}
Where as the connector generates the schema with the name as `record` hardcoded and no namespace. The below schema is generated by the connector. I'm expecting the connector should pull the schema from the subject and use ConfluentAvroRowSerialization to[which is not there today i believe] serialize using the schema from the subject.
{quote}{"type":"record",{color:#ff0000}"name":"record"{color},"fields":[\\{"name":"customerId","type":"string"},\\{"name":"age","type":"int"}]}
{quote}
and it throws "Failed to serialize row"
!Screen Shot 2021-10-12 at 10.38.55 AM.png|width=501,height=142!
I believe I should close this bug and create a new one, If its a bug
was (Author: mahen):
Thank you [~MartijnVisser], I figured the issue is not with the default, it is with the schema name that is generated by the connector.
The subjects has the below schema registered
{quote}{"type":"record","name":"SimpleCustomer","namespace":"com.nordstrom.nap.onehop.example.model","fields":[\{"name":"customerId","type":"string"},\{"name":"age","type":"int","default":0}]}
{quote}
Where as the connector generates the schema with the name as `record` hardcoded and no namespace. The below schema is generated by the connector. I'm expecting the connector should pull the schema from the subject and use ConfluentAvroRowSerialization to[which is not there today i believe] serialize using the schema from the subject.
{quote}{"type":"record",{color:#FF0000}"name":"record"{color},"fields":[\{"name":"customerId","type":"string"},\{"name":"age","type":"int"}]}
{quote}
and it throws "Failed to serialize row"
!Screen Shot 2021-10-12 at 10.38.55 AM.png|width=501,height=142!
I believe I should close this bug and create a new one, If its a bug
> Avro Confluent Registry SQL not fails to write to the topic with schema with default value for int
> --------------------------------------------------------------------------------------------------
>
> Key: FLINK-24494
> URL: https://issues.apache.org/jira/browse/FLINK-24494
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka, Table SQL / API
> Affects Versions: 1.13.2
> Reporter: Mahendran Ponnusamy
> Priority: Critical
> Attachments: KafkaTableTumbling.java, Screen Shot 2021-10-12 at 10.38.55 AM.png
>
>
>
> Given a schema registered to a topic with default value as '0' for int,
> when the flink sql with upsert-kafka connector writes to the topic,
> it fails coz the schema registered is not compatible with the data [schema] it is producing where there is not default value [{color:#9876aa}"default" {color}{color:#cc7832}: {color}{color:#6897bb}0{color}] for the int.
> {color:#cc7832} {color}
> {
> {color:#9876aa}"type" {color}{color:#cc7832}: {color}{color:#6a8759}"record"{color}{color:#cc7832},{color} {color:#9876aa}"name" {color}{color:#cc7832}: {color}{color:#6a8759}"SimpleCustomer"{color}{color:#cc7832},{color} {color:#9876aa}"namespace" {color}{color:#cc7832}: {color}{color:#6a8759}"com.nordstrom.nap.onehop.example.model"{color}{color:#cc7832},{color} {color:#9876aa}"fields" {color}{color:#cc7832}: {color}[ {
> {color:#9876aa}"name" {color}{color:#cc7832}: {color}{color:#6a8759}"customerId"{color}{color:#cc7832},{color} {color:#9876aa}"type" {color}{color:#cc7832}: {color}{color:#6a8759}"string"{color} }{color:#cc7832}, {color}{
> {color:#9876aa}"name" {color}{color:#cc7832}: {color}{color:#6a8759}"age"{color}{color:#cc7832},{color} {color:#9876aa}"type" {color}{color:#cc7832}: {color}{color:#6a8759}"int"{color}{color:#cc7832},{color} {color:#9876aa}"default" {color}{color:#cc7832}: {color}{color:#6897bb}0{color} }]
> }
>
> The full code is here [^KafkaTableTumbling.java]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)