You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Xuefu Zhang (JIRA)" <ji...@apache.org> on 2013/12/20 01:21:07 UTC
[jira] [Updated] (HIVE-6056) The AvroSerDe gives out
BadSchemaException if a partition is added to the table
[ https://issues.apache.org/jira/browse/HIVE-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xuefu Zhang updated HIVE-6056:
------------------------------
Description:
While creating an external table if I do not add a partition, I am able to read files using following format:
{code}
CREATE external TABLE event
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
LOCATION 's3n://test-event/input/2013/14/10'
TBLPROPERTIES ('avro.schema.literal' = '..some schema..');
{code}
but if I add a partition based on date
{code}
CREATE external TABLE event
PARTITIONED BY (ds STRING)
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
LOCATION 's3n://test-event/input/'
TBLPROPERTIES ('avro.schema.literal' = '..some schema..');
ALTER TABLE event ADD IF NOT EXISTS PARTITION (ds = '2013_12_16') LOCATION '2013/12/16/';
{code}
I get the following exception:
{code}
java.io.IOException:org.apache.hadoop.hive.serde2.avro.BadSchemaException
{code}
was:
While creating an external table if I do not add a partition, I am able to read files using following format:
CREATE external TABLE event
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
LOCATION 's3n://test-event/input/2013/14/10'
TBLPROPERTIES ('avro.schema.literal' = '..some schema..');
but if I add a partition based on date
CREATE external TABLE event
PARTITIONED BY (ds STRING)
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
LOCATION 's3n://test-event/input/'
TBLPROPERTIES ('avro.schema.literal' = '..some schema..');
ALTER TABLE event ADD IF NOT EXISTS PARTITION (ds = '2013_12_16') LOCATION '2013/12/16/';
I get the following exception:
java.io.IOException:org.apache.hadoop.hive.serde2.avro.BadSchemaException
> The AvroSerDe gives out BadSchemaException if a partition is added to the table
> -------------------------------------------------------------------------------
>
> Key: HIVE-6056
> URL: https://issues.apache.org/jira/browse/HIVE-6056
> Project: Hive
> Issue Type: Bug
> Components: Database/Schema
> Affects Versions: 0.11.0
> Environment: amazon EMR (hadoop Amazon 1.0.3), avro-1.7.5
> Reporter: Rushil Gupta
>
> While creating an external table if I do not add a partition, I am able to read files using following format:
> {code}
> CREATE external TABLE event
> ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
> STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
> OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
> LOCATION 's3n://test-event/input/2013/14/10'
> TBLPROPERTIES ('avro.schema.literal' = '..some schema..');
> {code}
> but if I add a partition based on date
> {code}
> CREATE external TABLE event
> PARTITIONED BY (ds STRING)
> ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
> STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
> OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
> LOCATION 's3n://test-event/input/'
> TBLPROPERTIES ('avro.schema.literal' = '..some schema..');
> ALTER TABLE event ADD IF NOT EXISTS PARTITION (ds = '2013_12_16') LOCATION '2013/12/16/';
> {code}
> I get the following exception:
> {code}
> java.io.IOException:org.apache.hadoop.hive.serde2.avro.BadSchemaException
> {code}
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)