[jira] [Created] (HIVE-7049) Unable to deserialize AVRO data when file schema and record schema are different and nullable

["Mohammad Kamrul Islam (JIRA)" <ji...@apache.org>]

Mohammad Kamrul Islam created HIVE-7049:
-------------------------------------------

             Summary: Unable to deserialize AVRO data when file schema and record schema are different and nullable
                 Key: HIVE-7049
                 URL: https://issues.apache.org/jira/browse/HIVE-7049
             Project: Hive
          Issue Type: Bug
            Reporter: Mohammad Kamrul Islam
            Assignee: Mohammad Kamrul Islam


It mainly happens when 
1 )file schema and record schema are not same
2 ) Record schema is nullable  but file schema is not.

The potential code location is at class AvroDeserialize
 
{noformat}
 if(AvroSerdeUtils.isNullableType(recordSchema)) {
      return deserializeNullableUnion(datum, fileSchema, recordSchema, columnType);
    }
{noformat}

In the above code snippet, recordSchema is verified if it is nullable. But the file schema is not checked.

I tested with these values:
{noformat}
recordSchema= ["null","string"]
fielSchema= "string"
{noformat}

And i got the following exception <line numbers might not be the same due to mu debugged code version>.

{noformat}
org.apache.avro.AvroRuntimeException: Not a union: "string" 
        at org.apache.avro.Schema.getTypes(Schema.java:272)
        at org.apache.hadoop.hive.serde2.avro.AvroDeserializer.deserializeNullableUnion(AvroDeserializer.java:275)
        at org.apache.hadoop.hive.serde2.avro.AvroDeserializer.worker(AvroDeserializer.java:205)
        at org.apache.hadoop.hive.serde2.avro.AvroDeserializer.workerBase(AvroDeserializer.java:188)
        at org.apache.hadoop.hive.serde2.avro.AvroDeserializer.deserialize(AvroDeserializer.java:174)
        at org.apache.hadoop.hive.serde2.avro.TestAvroDeserializer.verifyNullableType(TestAvroDeserializer.java:487)
        at org.apache.hadoop.hive.serde2.avro.TestAvroDeserializer.canDeserializeNullableTypes(TestAvroDeserializer.java:407)

{noformat}




--
This message was sent by Atlassian JIRA
(v6.2#6252)