You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Yubao Liu (Jira)" <ji...@apache.org> on 2021/08/13 10:39:00 UTC
[jira] [Updated] (AVRO-3186) Avro can't decode union field of a record
[ https://issues.apache.org/jira/browse/AVRO-3186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yubao Liu updated AVRO-3186:
----------------------------
Description:
[https://github.com/apache/avro/blob/release-1.10.2/lang/java/avro/src/main/java/org/apache/avro/generic/GenericDatumReader.java#L163]
{code:java}
protected Object read(Object old, Schema expected, ResolvingDecoder in) throws IOException {
Object datum = readWithoutConversion(old, expected, in);
LogicalType logicalType = expected.getLogicalType();
if (logicalType != null) {
Conversion<?> conversion = getData().getConversionFor(logicalType); // <---- bad
if (conversion != null) {
return convert(datum, expected, logicalType, conversion);
}
}
return datum;
}
{code}
For a logic type field in a union, getData().getConversionFor(logicalType) returns null, the cause is SpecificData.getForSchema() isn't compatible with JDK >= 9, it can't depend on Class.forName() to load user class.
[https://github.com/apache/avro/blob/release-1.10.2/lang/java/avro/src/main/java/org/apache/avro/specific/SpecificData.java#L147]
{code:java}
diff --git a/lang/java/avro/src/main/java/org/apache/avro/specific/SpecificData.java b/lang/java/avro/src/main/java/org/apache/avro/specific/SpecificData.java
index 58bc660b..33fc3418 100644
--- a/lang/java/avro/src/main/java/org/apache/avro/specific/SpecificData.java
+++ b/lang/java/avro/src/main/java/org/apache/avro/specific/SpecificData.java
@@ -140,15 +140,9 @@ public class SpecificData extends GenericData {
*/
public static SpecificData getForSchema(Schema reader) {
if (reader != null && reader.getType() == Type.RECORD) {
- final String className = getClassName(reader);
- if (className != null) {
- final Class<?> clazz;
- try {
- clazz = Class.forName(className);
- return getForClass(clazz);
- } catch (ClassNotFoundException e) {
- return SpecificData.get();
- }
+ final Class<?> clazz = SpecificData.get().getClass(reader);
+ if (clazz != null) {
+ return getForClass(clazz);
}
}
return SpecificData.get();
{code}
was:
[https://github.com/apache/avro/blob/release-1.10.2/lang/java/avro/src/main/java/org/apache/avro/generic/GenericDatumReader.java#L163]
{code:java}
protected Object read(Object old, Schema expected, ResolvingDecoder in) throws IOException {
Object datum = readWithoutConversion(old, expected, in);
LogicalType logicalType = expected.getLogicalType();
if (logicalType != null) {
Conversion<?> conversion = getData().getConversionFor(logicalType); // <---- bad
if (conversion != null) {
return convert(datum, expected, logicalType, conversion);
}
}
return datum;
}
{code}
For a logic type field in a union, getData().getConversionFor(logicalType) returns null, the cause is SpecificData.getForSchema() isn't compatible with JDK >= 9, it can't depend on Class.forName() to load user class.
[https://github.com/apache/avro/blob/release-1.10.2/lang/java/avro/src/main/java/org/apache/avro/specific/SpecificData.java#L147]
{code:diff}
diff --git a/lang/java/avro/src/main/java/org/apache/avro/specific/SpecificData.java b/lang/java/avro/src/main/java/org/apache/avro/specific/SpecificData.java
index 58bc660b..33fc3418 100644
--- a/lang/java/avro/src/main/java/org/apache/avro/specific/SpecificData.java
+++ b/lang/java/avro/src/main/java/org/apache/avro/specific/SpecificData.java
@@ -140,15 +140,9 @@ public class SpecificData extends GenericData {
*/
public static SpecificData getForSchema(Schema reader) {
if (reader != null && reader.getType() == Type.RECORD) {
- final String className = getClassName(reader);
- if (className != null) {
- final Class<?> clazz;
- try {
- clazz = Class.forName(className);
- return getForClass(clazz);
- } catch (ClassNotFoundException e) {
- return SpecificData.get();
- }
+ final Class<?> clazz = SpecificData.get().getClass(reader);
+ if (clazz != null) {
+ return getForClass(clazz);
}
}
return SpecificData.get();
{code}
> Avro can't decode union<null, logicalType> field of a record
> -------------------------------------------------------------
>
> Key: AVRO-3186
> URL: https://issues.apache.org/jira/browse/AVRO-3186
> Project: Apache Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.10.2
> Environment: JDK 11, macOS.
> Reporter: Yubao Liu
> Priority: Critical
>
> [https://github.com/apache/avro/blob/release-1.10.2/lang/java/avro/src/main/java/org/apache/avro/generic/GenericDatumReader.java#L163]
> {code:java}
> protected Object read(Object old, Schema expected, ResolvingDecoder in) throws IOException {
> Object datum = readWithoutConversion(old, expected, in);
> LogicalType logicalType = expected.getLogicalType();
> if (logicalType != null) {
> Conversion<?> conversion = getData().getConversionFor(logicalType); // <---- bad
> if (conversion != null) {
> return convert(datum, expected, logicalType, conversion);
> }
> }
> return datum;
> }
> {code}
>
> For a logic type field in a union, getData().getConversionFor(logicalType) returns null, the cause is SpecificData.getForSchema() isn't compatible with JDK >= 9, it can't depend on Class.forName() to load user class.
> [https://github.com/apache/avro/blob/release-1.10.2/lang/java/avro/src/main/java/org/apache/avro/specific/SpecificData.java#L147]
> {code:java}
> diff --git a/lang/java/avro/src/main/java/org/apache/avro/specific/SpecificData.java b/lang/java/avro/src/main/java/org/apache/avro/specific/SpecificData.java
> index 58bc660b..33fc3418 100644
> --- a/lang/java/avro/src/main/java/org/apache/avro/specific/SpecificData.java
> +++ b/lang/java/avro/src/main/java/org/apache/avro/specific/SpecificData.java
> @@ -140,15 +140,9 @@ public class SpecificData extends GenericData {
> */
> public static SpecificData getForSchema(Schema reader) {
> if (reader != null && reader.getType() == Type.RECORD) {
> - final String className = getClassName(reader);
> - if (className != null) {
> - final Class<?> clazz;
> - try {
> - clazz = Class.forName(className);
> - return getForClass(clazz);
> - } catch (ClassNotFoundException e) {
> - return SpecificData.get();
> - }
> + final Class<?> clazz = SpecificData.get().getClass(reader);
> + if (clazz != null) {
> + return getForClass(clazz);
> }
> }
> return SpecificData.get();
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)