You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by P lva <ru...@gmail.com> on 2014/01/15 19:25:28 UTC
Hive Import/Export
Hello,
I'm trying to move a hive table to a different cluster. My table is
partitioned. However when I use hive export-distcp-import, the partitions
are reversed. /warehouse/db/table/a/b/c in old cluster is
/warehouse/db/table/c/b/a in the new cluster.
How do I avoid this ?
Thanks
HIVE-5207 (Support data encryption for Hive tables)
Posted by "Subramanian, Sanjay (HQP)" <sa...@roberthalf.com>.
Hi guys
Looks like I have hit a stone wall with this feature not being there.
Specially since I am using Amazon EMR with Hive.
Thanks
Warm Regards
Sanjay
From: P lva <ru...@gmail.com>>
Reply-To: <us...@hive.apache.org>>
Date: Thu, 16 Jan 2014 11:09:30 -0600
To: <us...@hive.apache.org>>
Subject: Re: Hive Import/Export
This works fine too. Thanks a lot !
On Wed, Jan 15, 2014 at 5:17 PM, Stephen Sprague <sp...@gmail.com>> wrote:
wow. that's the craziest thing i've ever seen.
my approach to moving tables (partitioned or not) is:
1. run the ddl to create the table on the new cluster
2. distcp the hdfs data.
3. you might have to use "hdfs dfs -mv" on the new cluster to position your table properly under /warehouse/db depending on your distcp command.
4. run "msck repair table <table>" in hive to discover the partitions and populate the metastore accordingly.
works very well for me.
cheers,
Stephen.
PS no need to "export" or "import" data.
On Wed, Jan 15, 2014 at 10:25 AM, P lva <ru...@gmail.com>> wrote:
Hello,
I'm trying to move a hive table to a different cluster. My table is partitioned. However when I use hive export-distcp-import, the partitions are reversed. /warehouse/db/table/a/b/c in old cluster is /warehouse/db/table/c/b/a in the new cluster.
How do I avoid this ?
Thanks
Re: Hive Import/Export
Posted by P lva <ru...@gmail.com>.
This works fine too. Thanks a lot !
On Wed, Jan 15, 2014 at 5:17 PM, Stephen Sprague <sp...@gmail.com> wrote:
> wow. that's the craziest thing i've ever seen.
>
> my approach to moving tables (partitioned or not) is:
>
> 1. run the ddl to create the table on the new cluster
> 2. distcp the hdfs data.
> 3. you might have to use "hdfs dfs -mv" on the new cluster to position
> your table properly under /warehouse/db depending on your distcp command.
> 4. run "msck repair table <table>" in hive to discover the partitions and
> populate the metastore accordingly.
>
> works very well for me.
>
> cheers,
> Stephen.
> PS no need to "export" or "import" data.
>
>
>
> On Wed, Jan 15, 2014 at 10:25 AM, P lva <ru...@gmail.com> wrote:
>
>> Hello,
>>
>> I'm trying to move a hive table to a different cluster. My table is
>> partitioned. However when I use hive export-distcp-import, the partitions
>> are reversed. /warehouse/db/table/a/b/c in old cluster is
>> /warehouse/db/table/c/b/a in the new cluster.
>>
>> How do I avoid this ?
>>
>>
>> Thanks
>>
>>
>
>
Re: Hive Import/Export
Posted by Stephen Sprague <sp...@gmail.com>.
wow. that's the craziest thing i've ever seen.
my approach to moving tables (partitioned or not) is:
1. run the ddl to create the table on the new cluster
2. distcp the hdfs data.
3. you might have to use "hdfs dfs -mv" on the new cluster to position your
table properly under /warehouse/db depending on your distcp command.
4. run "msck repair table <table>" in hive to discover the partitions and
populate the metastore accordingly.
works very well for me.
cheers,
Stephen.
PS no need to "export" or "import" data.
On Wed, Jan 15, 2014 at 10:25 AM, P lva <ru...@gmail.com> wrote:
> Hello,
>
> I'm trying to move a hive table to a different cluster. My table is
> partitioned. However when I use hive export-distcp-import, the partitions
> are reversed. /warehouse/db/table/a/b/c in old cluster is
> /warehouse/db/table/c/b/a in the new cluster.
>
> How do I avoid this ?
>
>
> Thanks
>
>