You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by "Chagarlamudi, Prasanth" <pr...@epsilon.com> on 2015/06/18 20:10:58 UTC
Updating hive metadata
Hello,
Is there a way to update metadata in hive?
Created database mydb;
Created my_point_table;
Created my_managed_table;
Insert into my_managed_table from my_point_table;
Now,
Create my_point_table_2;
//Copy data from hive managed_table to managed_table_2's location
hdfs dfs -cp /user/hive/warehouse/mydb.db/my_managed_table /user/hive/warehouse/mydb.db/my_managed_table_2
At this point, I am expecting the following query
Select * from my_managed_table_2; to give me all the data I just copied from my_managed_table;
How do I update the hive metastore to consider the data that I copied to my_managed_table_2? Is that even possible?
Thanks in advance
Prasanth Chagarlamudi
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
RE: Updating hive metadata
Posted by "Chagarlamudi, Prasanth" <pr...@epsilon.com>.
Thank you Dev and Ryan.
Repair table worked. One last question. Are there any known side effects for repair table?
Thanks
Prasanth Chagarlamudi
From: Ryan Harris [mailto:Ryan.Harris@zionsbancorp.com]
Sent: Thursday, June 18, 2015 11:54 PM
To: user@hive.apache.org
Subject: RE: Updating hive metadata
you *should* be able to do:
create my_table_2 like my_table;
dfs -cp /user/hive/warehouse/my_table/* /user/hive/warehouse/my_table_2/
MSCK repair table my_table_2;
From: Devopam Mittra [mailto:devopam@gmail.com]<mailto:[mailto:devopam@gmail.com]>
Sent: Thursday, June 18, 2015 10:12 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: Updating hive metadata
hi Prasanth,
I would not suggest tweaking hive metastore info unless you have full knowledge of the entire tables that will get impacted due to such a change. And such things break a lot with upgrades since this is quite unmanageable manually.
Why don't you create my_managed_table_2 as type EXTERNAL and link it to the copied data in hdfs layer ..
regards
Dev
On Thu, Jun 18, 2015 at 11:40 PM, Chagarlamudi, Prasanth <pr...@epsilon.com>> wrote:
Hello,
Is there a way to update metadata in hive?
Created database mydb;
Created my_point_table;
Created my_managed_table;
Insert into my_managed_table from my_point_table;
Now,
Create my_point_table_2;
//Copy data from hive managed_table to managed_table_2’s location
hdfs dfs –cp /user/hive/warehouse/mydb.db/my_managed_table /user/hive/warehouse/mydb.db/my_managed_table_2
At this point, I am expecting the following query
Select * from my_managed_table_2; to give me all the data I just copied from my_managed_table;
How do I update the hive metastore to consider the data that I copied to my_managed_table_2? Is that even possible?
Thanks in advance
Prasanth Chagarlamudi
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
--
Devopam Mittra
Life and Relations are not binary
________________________________
THIS ELECTRONIC MESSAGE, INCLUDING ANY ACCOMPANYING DOCUMENTS, IS CONFIDENTIAL and may contain information that is privileged and exempt from disclosure under applicable law. If you are neither the intended recipient nor responsible for delivering the message to the intended recipient, please note that any dissemination, distribution, copying or the taking of any action in reliance upon the message is strictly prohibited. If you have received this communication in error, please notify the sender immediately. Thank you.
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
RE: Updating hive metadata
Posted by Ryan Harris <Ry...@zionsbancorp.com>.
you *should* be able to do:
create my_table_2 like my_table;
dfs -cp /user/hive/warehouse/my_table/* /user/hive/warehouse/my_table_2/
MSCK repair table my_table_2;
From: Devopam Mittra [mailto:devopam@gmail.com]
Sent: Thursday, June 18, 2015 10:12 PM
To: user@hive.apache.org
Subject: Re: Updating hive metadata
hi Prasanth,
I would not suggest tweaking hive metastore info unless you have full knowledge of the entire tables that will get impacted due to such a change. And such things break a lot with upgrades since this is quite unmanageable manually.
Why don't you create my_managed_table_2 as type EXTERNAL and link it to the copied data in hdfs layer ..
regards
Dev
On Thu, Jun 18, 2015 at 11:40 PM, Chagarlamudi, Prasanth <pr...@epsilon.com>> wrote:
Hello,
Is there a way to update metadata in hive?
Created database mydb;
Created my_point_table;
Created my_managed_table;
Insert into my_managed_table from my_point_table;
Now,
Create my_point_table_2;
//Copy data from hive managed_table to managed_table_2’s location
hdfs dfs –cp /user/hive/warehouse/mydb.db/my_managed_table /user/hive/warehouse/mydb.db/my_managed_table_2
At this point, I am expecting the following query
Select * from my_managed_table_2; to give me all the data I just copied from my_managed_table;
How do I update the hive metastore to consider the data that I copied to my_managed_table_2? Is that even possible?
Thanks in advance
Prasanth Chagarlamudi
________________________________
This e-mail and files transmitted with it are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not the intended recipient, or the employee or agent responsible to deliver it to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you are not one of the named recipient(s) or otherwise have reason to believe that you received this message in error, please immediately notify sender by e-mail, and destroy the original message. Thank You.
--
Devopam Mittra
Life and Relations are not binary
======================================================================
THIS ELECTRONIC MESSAGE, INCLUDING ANY ACCOMPANYING DOCUMENTS, IS CONFIDENTIAL and may contain information that is privileged and exempt from disclosure under applicable law. If you are neither the intended recipient nor responsible for delivering the message to the intended recipient, please note that any dissemination, distribution, copying or the taking of any action in reliance upon the message is strictly prohibited. If you have received this communication in error, please notify the sender immediately. Thank you.
Re: Updating hive metadata
Posted by Devopam Mittra <de...@gmail.com>.
hi Prasanth,
I would not suggest tweaking hive metastore info unless you have full
knowledge of the entire tables that will get impacted due to such a change.
And such things break a lot with upgrades since this is quite unmanageable
manually.
Why don't you create my_managed_table_2 as type EXTERNAL and link it to the
copied data in hdfs layer ..
regards
Dev
On Thu, Jun 18, 2015 at 11:40 PM, Chagarlamudi, Prasanth <
prasanth.chagarlamudi@epsilon.com> wrote:
> Hello,
>
> Is there a way to update metadata in hive?
>
>
>
> Created database *mydb*;
>
> Created *my_point_table*;
>
> Created *my_managed_table*;
>
> Insert into *my_managed_table* from *my_point_table*;
>
>
>
> Now,
>
> Create *my_point_table_2*;
>
> //Copy data from hive* managed_table *to* managed_table_2*’s location
>
> hdfs dfs –cp /user/hive/warehouse/mydb.db/*my_managed_table*
> /user/hive/warehouse/mydb.db/*my_managed_table_2*
>
> At this point, I am expecting the following query
>
> Select * from *my_managed_table_2*; to give me all the data I
> just copied from *my_managed_table*;
>
>
>
> How do I update the hive metastore to consider the data that I copied to
> *my_managed_table_2*? Is that even possible?
>
>
>
> Thanks in advance
>
> Prasanth Chagarlamudi
>
>
>
>
>
> ------------------------------
>
> This e-mail and files transmitted with it are confidential, and are
> intended solely for the use of the individual or entity to whom this e-mail
> is addressed. If you are not the intended recipient, or the employee or
> agent responsible to deliver it to the intended recipient, you are hereby
> notified that any dissemination, distribution or copying of this
> communication is strictly prohibited. If you are not one of the named
> recipient(s) or otherwise have reason to believe that you received this
> message in error, please immediately notify sender by e-mail, and destroy
> the original message. Thank You.
>
--
Devopam Mittra
Life and Relations are not binary