You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "xiaozilong (Jira)" <ji...@apache.org> on 2020/10/09 10:00:00 UTC
[jira] [Updated] (FLINK-19291) Meeting `SchemaParseException` when
use `AvroSchemaConverter` converts flink logical type
[ https://issues.apache.org/jira/browse/FLINK-19291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
xiaozilong updated FLINK-19291:
-------------------------------
Description:
Hi, I use `AvroSchemaConverter` to converts flink logical type into an avro schema will be throw `SchemaParseException` when there are both row type and row array type
This is my simplified code
{code:java}
RowType rowType = (RowType) TableSchema.builder()
.field("row", DataTypes.ROW(DataTypes.FIELD("a", DataTypes.STRING())))
.field("row_array", DataTypes.ROW(DataTypes.FIELD("b", DataTypes.STRING())))
.build().toRowDataType().getLogicalType();
Schema schema = AvroSchemaConverter.convertToSchema(rowType);
System.out.println(schema);
{code}
it will throw
{code:java}
org.apache.avro.SchemaParseException: Can't redefine: row_1
at org.apache.avro.Schema$Names.put(Schema.java:1128)
at org.apache.avro.Schema$NamedSchema.writeNameRef(Schema.java:562)
at org.apache.avro.Schema$RecordSchema.toJson(Schema.java:690)
at org.apache.avro.Schema$ArraySchema.toJson(Schema.java:805)
at org.apache.avro.Schema$UnionSchema.toJson(Schema.java:882)
at org.apache.avro.Schema$RecordSchema.fieldsToJson(Schema.java:716)
at org.apache.avro.Schema$RecordSchema.toJson(Schema.java:701)
at org.apache.avro.Schema.toString(Schema.java:324)
at org.apache.avro.Schema.toString(Schema.java:314)
at java.lang.String.valueOf(String.java:2994)
at java.io.PrintStream.println(PrintStream.java:821)
at org.apache.flink.formats.avro.typeutils.AvroSchemaConverterTest.testRowTypeAvroSchemaConversion(AvroSchemaConverterTest.java:99)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
{code}
Thanks!
was:
Hi, I use `AvroSchemaConverter` to converts flink logical type into an avro schema will be throw `SchemaParseException` when there are both row type and row array type
This is my simplified code
{code:java}
RowType rowType = (RowType) TableSchema.builder()
.field("row", DataTypes.ROW(DataTypes.FIELD("string_field", DataTypes.STRING())))
.field("row_array", DataTypes.ARRAY(DataTypes.ROW(DataTypes.FIELD("string_field", DataTypes.STRING()))))
.build().toRowDataType().getLogicalType();
Schema schema = AvroSchemaConverter.convertToSchema(rowType);
System.out.println(schema);
{code}
it will throw
{code:java}
org.apache.avro.SchemaParseException: Can't redefine: row_1
at org.apache.avro.Schema$Names.put(Schema.java:1128)
at org.apache.avro.Schema$NamedSchema.writeNameRef(Schema.java:562)
at org.apache.avro.Schema$RecordSchema.toJson(Schema.java:690)
at org.apache.avro.Schema$ArraySchema.toJson(Schema.java:805)
at org.apache.avro.Schema$UnionSchema.toJson(Schema.java:882)
at org.apache.avro.Schema$RecordSchema.fieldsToJson(Schema.java:716)
at org.apache.avro.Schema$RecordSchema.toJson(Schema.java:701)
at org.apache.avro.Schema.toString(Schema.java:324)
at org.apache.avro.Schema.toString(Schema.java:314)
at java.lang.String.valueOf(String.java:2994)
at java.io.PrintStream.println(PrintStream.java:821)
at org.apache.flink.formats.avro.typeutils.AvroSchemaConverterTest.testRowTypeAvroSchemaConversion(AvroSchemaConverterTest.java:99)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
{code}
Thanks!
> Meeting `SchemaParseException` when use `AvroSchemaConverter` converts flink logical type
> -----------------------------------------------------------------------------------------
>
> Key: FLINK-19291
> URL: https://issues.apache.org/jira/browse/FLINK-19291
> Project: Flink
> Issue Type: Bug
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
> Affects Versions: 1.11.0
> Reporter: xiaozilong
> Assignee: xiaozilong
> Priority: Major
> Labels: pull-request-available
> Attachments: image-2020-09-22-15-16-50-130.png
>
>
> Hi, I use `AvroSchemaConverter` to converts flink logical type into an avro schema will be throw `SchemaParseException` when there are both row type and row array type
>
> This is my simplified code
> {code:java}
> RowType rowType = (RowType) TableSchema.builder()
> .field("row", DataTypes.ROW(DataTypes.FIELD("a", DataTypes.STRING())))
> .field("row_array", DataTypes.ROW(DataTypes.FIELD("b", DataTypes.STRING())))
> .build().toRowDataType().getLogicalType();
> Schema schema = AvroSchemaConverter.convertToSchema(rowType);
> System.out.println(schema);
> {code}
>
> it will throw
> {code:java}
> org.apache.avro.SchemaParseException: Can't redefine: row_1
> at org.apache.avro.Schema$Names.put(Schema.java:1128)
> at org.apache.avro.Schema$NamedSchema.writeNameRef(Schema.java:562)
> at org.apache.avro.Schema$RecordSchema.toJson(Schema.java:690)
> at org.apache.avro.Schema$ArraySchema.toJson(Schema.java:805)
> at org.apache.avro.Schema$UnionSchema.toJson(Schema.java:882)
> at org.apache.avro.Schema$RecordSchema.fieldsToJson(Schema.java:716)
> at org.apache.avro.Schema$RecordSchema.toJson(Schema.java:701)
> at org.apache.avro.Schema.toString(Schema.java:324)
> at org.apache.avro.Schema.toString(Schema.java:314)
> at java.lang.String.valueOf(String.java:2994)
> at java.io.PrintStream.println(PrintStream.java:821)
> at org.apache.flink.formats.avro.typeutils.AvroSchemaConverterTest.testRowTypeAvroSchemaConversion(AvroSchemaConverterTest.java:99)
> at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
> {code}
> Thanks!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)