[jira] [Commented] (HIVE-6308) COLUMNS_V2 Metastore table not populated for tables created without an explicit column list.

["Julien Lehuen (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-6308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14071422#comment-14071422 ] 

Julien Lehuen commented on HIVE-6308:
-------------------------------------

This is *critical* for Impala users! We want to be able to perform our Metastore operations via Hive's Thrift server, but this would create unhealthy tables for which we can not run ANALYZE. Such table's statistics cannot computed by Impala (statistics are absolutely necessary for Impala to run joins in an optimal manner), and this means that we need to create table via Impala over JDBC. This is less than ideal, and this issue is the only blocker from letting us create via Hive over Thrift.

Related IMPALA tickets: [IMPALA-867|https://issues.cloudera.org/browse/IMPALA-867], [IMPALA-1104|https://issues.cloudera.org/browse/IMPALA-1104]

> COLUMNS_V2 Metastore table not populated for tables created without an explicit column list.
> --------------------------------------------------------------------------------------------
>
>                 Key: HIVE-6308
>                 URL: https://issues.apache.org/jira/browse/HIVE-6308
>             Project: Hive
>          Issue Type: Bug
>          Components: Database/Schema
>    Affects Versions: 0.10.0
>            Reporter: Alexander Behm
>
> Consider this example table:
> CREATE TABLE avro_test
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
> STORED as INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
> TBLPROPERTIES (
> 'avro.schema.url'='file:///path/to/the/schema/test_serializer.avsc');
> When I try to run an ANALYZE TABLE for computing column stats on any of the columns, then I get:
> org.apache.hadoop.hive.ql.metadata.HiveException: NoSuchObjectException(message:Column o_orderpriority for which stats gathering is requested doesn't exist.)
>     at org.apache.hadoop.hive.ql.metadata.Hive.updateTableColumnStatistics(Hive.java:2280)
>     at org.apache.hadoop.hive.ql.exec.ColumnStatsTask.persistTableStats(ColumnStatsTask.java:331)
>     at org.apache.hadoop.hive.ql.exec.ColumnStatsTask.execute(ColumnStatsTask.java:343)
>     at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:138)
>     at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:66)
>     at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1383)
>     at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1169)
>     at org.apache.hadoop.hive.ql.Driver.run(Driver.java:982)
>     at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902)
>     at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
>     at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
>     at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
>     at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
>     at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:606)
>     at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> The root cause appears to be that the COLUMNS_V2 table in the Metastore isn't populated properly during the table creation.



--
This message was sent by Atlassian JIRA
(v6.2#6252)