You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Filip (Jira)" <ji...@apache.org> on 2022/07/19 14:11:00 UTC
[jira] [Updated] (AVRO-3583) Avro ReflectData API does not allow referencing external schemas through @AvroSchema
[ https://issues.apache.org/jira/browse/AVRO-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Filip updated AVRO-3583:
------------------------
Description:
For easier transitioning into an Avro-based project, the {{ReflectData}} API allows the generation of {{.avsc}} Schema files from POJOs.
However, currently it does not seem to be possible to reference external (from the point of view of the POJO) schemas when doing so. Particularly, the following case does not seem covered:
{code:java}
package com.acme;
public class Seller {
@AvroSchema("{\"type\":\"com.acme.Address\",\"name\":\"sellerAddress\"}")
private Address sellerAddress;
// getters/setters/constructors
}
// separate file
package com.acme;
public class Address {
private String street;
private String streetNumber;
// ...
// getters/setters/constructors
}{code}
trying to use e.g.:
{code:java}
ReflectData.AllowNull.get().getSchema(Seller.class)
{code}
currently throws the following exception:
{code:java}
org.apache.avro.SchemaParseException: Type not supported: com.acme.Address{code}
{*}Note{*}: this does not have anything to do with forward references (which are disallowed by Avro). The problem lies in {{org.apache.avro.reflect.ReflectData#createFieldSchema}} which contains the following code:
{code:java}
865 AvroSchema explicit = field.getAnnotation(AvroSchema.class);
866 if (explicit != null) // explicit schema
867 return new Schema.Parser().parse(explicit.value()); {code}
which explicitly (no pun intended) creates a new {{Parser}} instance which is *not aware* of any other schema definitions that may have been processed previously.
Not using {{@AvroSchema}} is obviously a possibility, but:
# Any POJO with more than a few fields will be needlessly complex and long,
# Care must be taken *not to reparse* the referenced class, since it would lead to a redefinition of the schema and fail.
Is this intended behaviour? Is there a better way to generate schema files from existing POJOs for non-trivial classes hierarchies?
was:
For easier transitioning into an Avro-based project, the {{ReflectData}} API allows the generation of {{.avsc}} Schema files from POJOs.
However, currently it does not seem to be possible to reference external (from the point of view of the POJO) schemas when doing so. Particularly, the following case does not seem covered:
{code:java}
package com.acme;
public class Seller {
@AvroSchema("{\"type\":\"com.acme.Address\",\"name\":\"sellerAddress\"}")
private Address sellerAddress;
// getters/setters/constructors
}
// separate file
package com.acme;
public class Address {
private String street;
private String streetNumber;
// ...
// getters/setters/constructors
}{code}
trying to use e.g.:
{code:java}
ReflectData.AllowNull.get().getSchema(Seller.class)
{code}
currently throws the following exception:
{code:java}
org.apache.avro.SchemaParseException: Type not supported: com.acme.Seller {code}
{*}Note{*}: this does not have anything to do with forward references (which are disallowed by Avro). The problem lies in {{org.apache.avro.reflect.ReflectData#createFieldSchema}} which contains the following code:
{code:java}
865 AvroSchema explicit = field.getAnnotation(AvroSchema.class);
866 if (explicit != null) // explicit schema
867 return new Schema.Parser().parse(explicit.value()); {code}
which explicitly (no pun intended) creates a new {{Parser}} instance which is *not aware* of any other schema definitions that may have been processed previously.
Not using {{@AvroSchema}} is obviously a possibility, but:
# Any POJO with more than a few fields will be needlessly complex and long,
# Care must be taken *not to reparse* the referenced class, since it would lead to a redefinition of the schema and fail.
Is this intended behaviour? Is there a better way to generate schema files from existing POJOs for non-trivial classes hierarchies?
> Avro ReflectData API does not allow referencing external schemas through @AvroSchema
> ------------------------------------------------------------------------------------
>
> Key: AVRO-3583
> URL: https://issues.apache.org/jira/browse/AVRO-3583
> Project: Apache Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.10.2
> Reporter: Filip
> Priority: Major
>
> For easier transitioning into an Avro-based project, the {{ReflectData}} API allows the generation of {{.avsc}} Schema files from POJOs.
> However, currently it does not seem to be possible to reference external (from the point of view of the POJO) schemas when doing so. Particularly, the following case does not seem covered:
> {code:java}
> package com.acme;
> public class Seller {
>
> @AvroSchema("{\"type\":\"com.acme.Address\",\"name\":\"sellerAddress\"}")
> private Address sellerAddress;
> // getters/setters/constructors
> }
> // separate file
> package com.acme;
> public class Address {
>
> private String street;
> private String streetNumber;
> // ...
> // getters/setters/constructors
> }{code}
> trying to use e.g.:
> {code:java}
> ReflectData.AllowNull.get().getSchema(Seller.class)
> {code}
> currently throws the following exception:
> {code:java}
> org.apache.avro.SchemaParseException: Type not supported: com.acme.Address{code}
> {*}Note{*}: this does not have anything to do with forward references (which are disallowed by Avro). The problem lies in {{org.apache.avro.reflect.ReflectData#createFieldSchema}} which contains the following code:
> {code:java}
> 865 AvroSchema explicit = field.getAnnotation(AvroSchema.class);
> 866 if (explicit != null) // explicit schema
> 867 return new Schema.Parser().parse(explicit.value()); {code}
> which explicitly (no pun intended) creates a new {{Parser}} instance which is *not aware* of any other schema definitions that may have been processed previously.
> Not using {{@AvroSchema}} is obviously a possibility, but:
> # Any POJO with more than a few fields will be needlessly complex and long,
> # Care must be taken *not to reparse* the referenced class, since it would lead to a redefinition of the schema and fail.
> Is this intended behaviour? Is there a better way to generate schema files from existing POJOs for non-trivial classes hierarchies?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)