[jira] [Created] (HIVE-6589) Automatically add partitions for external tables

["Ken Dallmeyer (JIRA)" <ji...@apache.org>]

Ken Dallmeyer created HIVE-6589:
-----------------------------------

             Summary: Automatically add partitions for external tables
                 Key: HIVE-6589
                 URL: https://issues.apache.org/jira/browse/HIVE-6589
             Project: Hive
          Issue Type: New Feature
    Affects Versions: 0.10.0
            Reporter: Ken Dallmeyer


I have a data stream being loaded into Hadoop via Flume. It loads into a date partition folder in HDFS.  The path looks like this:
{code}/flume/my_data/YYYY/MM/DD/HH
/flume/my_data/2014/03/02/01
/flume/my_data/2014/03/02/02
/flume/my_data/2014/03/02/03{code}

On top of it I create an EXTERNAL hive table to do querying.  As of now, I have to manually add partitions.  What I want is for EXTERNAL tables, Hive should "discover" those partitions.  Additionally I would like to specify a partition pattern so that when I query Hive will know to use the partition pattern to find the HDFS folder.

So something like this:
{code}CREATE EXTERNAL TABLE my_data (
  col1 STRING,
  col2 INT
)
PARTITIONED BY (
  dt STRING,
  hour STRING
)
LOCATION 
  '/flume/mydata'
TBLPROPERTIES (
  'hive.partition.spec' = 'dt=$Y-$M-$D, hour=$H',
  'hive.partition.spec.location' = '$Y/$M/$D/$H',
);
{code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)