[jira] [Created] (HIVE-7174) Do not accept string as scale and precision when reading Avro schema

["Jarek Jarcec Cecho (JIRA)" <ji...@apache.org>]

Jarek Jarcec Cecho created HIVE-7174:
----------------------------------------

             Summary: Do not accept string as scale and precision when reading Avro schema
                 Key: HIVE-7174
                 URL: https://issues.apache.org/jira/browse/HIVE-7174
             Project: Hive
          Issue Type: Bug
            Reporter: Jarek Jarcec Cecho
            Assignee: Jarek Jarcec Cecho


I've noticed that the current AvroSerde will happily accept schema that uses string instead of integer for scale and precision, e.g. fragment {{"precision":"4","scale":"1"}} from following table:

{code}
CREATE TABLE `avro_dec1`(
  `name` string COMMENT 'from deserializer',
  `value` decimal(4,1) COMMENT 'from deserializer')
COMMENT 'just drop the schema right into the HQL'
ROW FORMAT SERDE
  'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
STORED AS INPUTFORMAT
  'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
TBLPROPERTIES (
  'numFiles'='1',
  'avro.schema.literal'='{\"namespace\":\"com.howdy\",\"name\":\"some_schema\",\"type\":\"record\",\"fields\":[{\"name\":\"name\",\"type\":\"string\"},{\"name\":\"value\",\"type\":{\"type\":\"bytes\",\"logicalType\":\"decimal\",\"precision\":\"4\",\"scale\":\"1\"}}]}'
);
{code}

However the Decimal spec defined in AVRO-1402 requires only integer to be there and hence is allowing only following fragment instead {{"precision":4,"scale":1}} (e.g. no double quotes around numbers).

As Hive can propagate this incorrect schema to new files and hence creating files with invalid schema, I think that we should alter the behavior and insist on the correct schema.



--
This message was sent by Atlassian JIRA
(v6.2#6252)