You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Szehon Ho (JIRA)" <ji...@apache.org> on 2015/04/04 02:15:34 UTC
[jira] [Commented] (HIVE-10206) Cannot modify Avro table properties
if existing properties are invalid
[ https://issues.apache.org/jira/browse/HIVE-10206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14395389#comment-14395389 ]
Szehon Ho commented on HIVE-10206:
----------------------------------
Sorry this is actually fixed by HIVE-2573 which for some reason changes org/apache/hadoop/hive/ql/metadata/Table/isField method to catch all exceptions (it turns out this is a RuntimeException thrown from AvroSerde). I was using Hive 1.1, which let the exception go to the top.
I think it is ok, but when doing this investigation, we might be able to improve it by not even calling getCols() for alter table and thus avoiding this exception if we are just altering some properties. It may save some performance.
> Cannot modify Avro table properties if existing properties are invalid
> ----------------------------------------------------------------------
>
> Key: HIVE-10206
> URL: https://issues.apache.org/jira/browse/HIVE-10206
> Project: Hive
> Issue Type: Bug
> Components: File Formats
> Affects Versions: 1.1.0
> Reporter: Szehon Ho
> Assignee: Ferdinand Xu
>
> Create an avro table with an external avsc file like:
> {noformat}
> CREATE TABLE test(...)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
> TBLPROPERTIES (
> 'avro.schema.url'='file:///Users/szehon/Temp/test.avsc',
> 'kite.compression.type'='snappy',
> 'transient_lastDdlTime'='1427996456')
> {noformat}
> Delete test.avsc file.
> Try to modify the table properties:
> {noformat}
> alter table test set tblproperties ('avro.schema.url'='file:///Users/szehon/Temp/test2.avsc');
> {noformat}
> Will throw an exception like AvroSerdeException:
> {noformat}
> at org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils.determineSchemaOrThrowException(AvroSerdeUtils.java:119)
> at org.apache.hadoop.hive.serde2.avro.AvroSerDe.determineSchemaOrReturnErrorSchema(AvroSerDe.java:163)
> at org.apache.hadoop.hive.serde2.avro.AvroSerDe.initialize(AvroSerDe.java:101)
> at org.apache.hadoop.hive.serde2.avro.AvroSerDe.initialize(AvroSerDe.java:78)
> at org.apache.hadoop.hive.serde2.SerDeUtils.initializeSerDe(SerDeUtils.java:520)
> at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:377)
> at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:274)
> at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:256)
> at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:595)
> at org.apache.hadoop.hive.ql.exec.DDLTask.alterTableOrSinglePartition(DDLTask.java:3383)
> at org.apache.hadoop.hive.ql.exec.DDLTask.alterTable(DDLTask.java:3340)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:332)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1638)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1397)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1181)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1047)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1037)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:207)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:159)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:370)
> at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:754)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:615)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java:-1)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)