You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Leo Alekseyev <dn...@gmail.com> on 2010/12/16 00:03:11 UTC
Organizing Hive warehouse with lots of tables
I have a lot of tables floating in /user/hive/warehouse. A lot of
them get generated in the process of doing some one-off analysis and
can safely be dropped; often, however, these tables aren't egregiously
huge and I'd like to keep them around. It would help if I could e.g.
organize my warehouse into per-project "sub-warehouses".
This seems like a common situation, so I was wondering if others have
come up with a clever way to manage Hive tables like that. I know you
can specify location in the CREATE TABLE statement; is this the best
way to do it?..
--Leo
Re: Organizing Hive warehouse with lots of tables
Posted by Edward Capriolo <ed...@gmail.com>.
On Wed, Dec 15, 2010 at 6:21 PM, Appan Thirumaligai
<at...@ngmoco.com> wrote:
> Use Create External Table statement like mentioned below and move the files to the folders of your choice in HDFS.
>
>
> On Dec 15, 2010, at 3:03 PM, Leo Alekseyev wrote:
>
>> I have a lot of tables floating in /user/hive/warehouse. A lot of
>> them get generated in the process of doing some one-off analysis and
>> can safely be dropped; often, however, these tables aren't egregiously
>> huge and I'd like to keep them around. It would help if I could e.g.
>> organize my warehouse into per-project "sub-warehouses".
>>
>> This seems like a common situation, so I was wondering if others have
>> come up with a clever way to manage Hive tables like that. I know you
>> can specify location in the CREATE TABLE statement; is this the best
>> way to do it?..
>>
>> --Leo
>
>
>
>
Hive 0.6.0 supports multiple schema/catalogs now
https://issues.apache.org/jira/browse/HIVE-675
Right now the catalog lives under /user/hive/warehouse but soon the
location can be specified at creation time.
Re: Organizing Hive warehouse with lots of tables
Posted by Appan Thirumaligai <at...@ngmoco.com>.
Use Create External Table statement like mentioned below and move the files to the folders of your choice in HDFS.
On Dec 15, 2010, at 3:03 PM, Leo Alekseyev wrote:
> I have a lot of tables floating in /user/hive/warehouse. A lot of
> them get generated in the process of doing some one-off analysis and
> can safely be dropped; often, however, these tables aren't egregiously
> huge and I'd like to keep them around. It would help if I could e.g.
> organize my warehouse into per-project "sub-warehouses".
>
> This seems like a common situation, so I was wondering if others have
> come up with a clever way to manage Hive tables like that. I know you
> can specify location in the CREATE TABLE statement; is this the best
> way to do it?..
>
> --Leo