You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Mohammad Kamrul Islam (JIRA)" <ji...@apache.org> on 2014/05/13 02:31:18 UTC
[jira] [Updated] (HIVE-7049) Unable to deserialize AVRO data when
file schema and record schema are different and nullable
[ https://issues.apache.org/jira/browse/HIVE-7049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mohammad Kamrul Islam updated HIVE-7049:
----------------------------------------
Attachment: HIVE-7049.1.patch
patch uploaded
> Unable to deserialize AVRO data when file schema and record schema are different and nullable
> ---------------------------------------------------------------------------------------------
>
> Key: HIVE-7049
> URL: https://issues.apache.org/jira/browse/HIVE-7049
> Project: Hive
> Issue Type: Bug
> Reporter: Mohammad Kamrul Islam
> Assignee: Mohammad Kamrul Islam
> Attachments: HIVE-7049.1.patch
>
>
> It mainly happens when
> 1 )file schema and record schema are not same
> 2 ) Record schema is nullable but file schema is not.
> The potential code location is at class AvroDeserialize
>
> {noformat}
> if(AvroSerdeUtils.isNullableType(recordSchema)) {
> return deserializeNullableUnion(datum, fileSchema, recordSchema, columnType);
> }
> {noformat}
> In the above code snippet, recordSchema is verified if it is nullable. But the file schema is not checked.
> I tested with these values:
> {noformat}
> recordSchema= ["null","string"]
> fielSchema= "string"
> {noformat}
> And i got the following exception <line numbers might not be the same due to mu debugged code version>.
> {noformat}
> org.apache.avro.AvroRuntimeException: Not a union: "string"
> at org.apache.avro.Schema.getTypes(Schema.java:272)
> at org.apache.hadoop.hive.serde2.avro.AvroDeserializer.deserializeNullableUnion(AvroDeserializer.java:275)
> at org.apache.hadoop.hive.serde2.avro.AvroDeserializer.worker(AvroDeserializer.java:205)
> at org.apache.hadoop.hive.serde2.avro.AvroDeserializer.workerBase(AvroDeserializer.java:188)
> at org.apache.hadoop.hive.serde2.avro.AvroDeserializer.deserialize(AvroDeserializer.java:174)
> at org.apache.hadoop.hive.serde2.avro.TestAvroDeserializer.verifyNullableType(TestAvroDeserializer.java:487)
> at org.apache.hadoop.hive.serde2.avro.TestAvroDeserializer.canDeserializeNullableTypes(TestAvroDeserializer.java:407)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)