[jira] [Created] (HIVE-13979) MSCK throws exceptions when unpartitioned table has nested directories

["Rajesh Balamohan (JIRA)" <ji...@apache.org>]

Rajesh Balamohan created HIVE-13979:
---------------------------------------

             Summary: MSCK throws exceptions when unpartitioned table has nested directories
                 Key: HIVE-13979
                 URL: https://issues.apache.org/jira/browse/HIVE-13979
             Project: Hive
          Issue Type: Bug
          Components: Metastore
            Reporter: Rajesh Balamohan
            Priority: Minor


When nested directories are present in unpartitioned dataset, msck throws exceptions. Query runs without issues though.

{noformat}
DataSet arranged as follows

tpch_flat_orc_1000.db/lineitem_random/a
tpch_flat_orc_1000.db/lineitem_random/b
tpch_flat_orc_1000.db/lineitem_random/c
tpch_flat_orc_1000.db/lineitem_random/d
tpch_flat_orc_1000.db/lineitem_random/e
tpch_flat_orc_1000.db/lineitem_random/f
tpch_flat_orc_1000.db/lineitem_random/g
tpch_flat_orc_1000.db/lineitem_random/h
tpch_flat_orc_1000.db/lineitem_random/i
tpch_flat_orc_1000.db/lineitem_random/j
tpch_flat_orc_1000.db/lineitem_random/k
tpch_flat_orc_1000.db/lineitem_random/l
tpch_flat_orc_1000.db/lineitem_random/m
tpch_flat_orc_1000.db/lineitem_random/n
tpch_flat_orc_1000.db/lineitem_random/o
tpch_flat_orc_1000.db/lineitem_random/p
tpch_flat_orc_1000.db/lineitem_random/q
tpch_flat_orc_1000.db/lineitem_random/r
tpch_flat_orc_1000.db/lineitem_random/s
tpch_flat_orc_1000.db/lineitem_random/t
tpch_flat_orc_1000.db/lineitem_random/u
tpch_flat_orc_1000.db/lineitem_random/v
tpch_flat_orc_1000.db/lineitem_random/w
tpch_flat_orc_1000.db/lineitem_random/x
tpch_flat_orc_1000.db/lineitem_random/y
tpch_flat_orc_1000.db/lineitem_random/z


CREATE EXTERNAL TABLE `lineitem_random`(
  `l_orderkey` bigint,
  `l_partkey` bigint,
  `l_suppkey` bigint,
  `l_linenumber` int,
  `l_quantity` double,
  `l_extendedprice` double,
  `l_discount` double,
  `l_tax` double,
  `l_returnflag` string,
  `l_linestatus` string,
  `l_shipdate` string,
  `l_commitdate` string,
  `l_receiptdate` string,
  `l_shipinstruct` string,
  `l_shipmode` string,
  `l_comment` string) stored as ORC LOCATION '.../tpch_flat_orc_1000.db/lineitem_random'

hive> SELECT sum(l_extendedprice * l_discount) AS revenue
    > FROM lineitem_random
    > WHERE l_shipdate >= '1993-01-01'
    >   AND l_shipdate < '1994-01-01'
    >   AND l_discount BETWEEN 0.06 - 0.01 AND 0.06 + 0.01
    >   AND l_quantity < 25;
....
....
Status: DAG finished successfully

...
...
msck repair table lineitem_random;
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask


2016-06-09 11:14:55,102 WARN  [main]: exec.DDLTask (DDLTask.java:msck(1787)) - Failed to run metacheck:
org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Unexpected component a)
        at org.apache.hadoop.hive.ql.exec.DDLTask.msck(DDLTask.java:1753)
        at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:375)
        at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
        at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89)
        at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1728)
        at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1485)
        at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1262)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1126)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1116)
        at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:216)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:168)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:379)
        at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:739)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:624)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: MetaException(message:Unexpected component a)
        at org.apache.hadoop.hive.metastore.Warehouse.makeValsFromName(Warehouse.java:390)
        at org.apache.hadoop.hive.ql.exec.DDLTask.msck(DDLTask.java:1751)
        ... 20 more  

{noformat}






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)