subdirectories in External tables.

[Sam William <sa...@stumbleupon.com>]

I have a case where  I want to  dump data daily under 

/user/sampd/test/{date}.


I thought of making this an external table on Hive   so that I   can just keep dumping data daily.   

In my external table definition , I specified  the location as /user/sampd/test. 


But when I try to run a query on the table , I get an exception .


Failed with exception java.io.IOException:java.io.IOException: Not a file:<hdfs path to the folder>.

This is inspite of setting mapred.input.dir.recursive=true;

Is this a supported feature in Hive ?   Any alternatives ?


Sam William
sampd@stumbleupon.com

Re: subdirectories in External tables.

[Mapred Learn <ma...@gmail.com>]

U shud create partitions by date and add partitions by alter table cmnd daily
U can then query data by date partition


Sent from my iPhone

On Aug 12, 2011, at 1:16 PM, Sam William <sa...@stumbleupon.com> wrote:

> 
> I have a case where  I want to  dump data daily under 
> 
> /user/sampd/test/{date}.
> 
> 
> I thought of making this an external table on Hive   so that I   can just keep dumping data daily.   
> 
> In my external table definition , I specified  the location as /user/sampd/test. 
> 
> 
> But when I try to run a query on the table , I get an exception .
> 
> 
> Failed with exception java.io.IOException:java.io.IOException: Not a file:<hdfs path to the folder>.
> 
> This is inspite of setting mapred.input.dir.recursive=true;
> 
> Is this a supported feature in Hive ?   Any alternatives ?
> 
> 
> Sam William
> sampd@stumbleupon.com
> 
> 
>