You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Devopam Mittra <de...@gmail.com> on 2015/01/16 15:09:18 UTC
Hive UPDATE : query around usage
hi All,
I wanted to UPDATE multiple rows in a table by joining it with another
table for identification of matching records (otherwise in the RDBMS world
also commonly known as SET based UPDATE).
The idea is to update multiple records in one go , to downsize the lead
time for each MR job to kick off.
Is it supported or I am being greedy here with my wishlist ?
Also, at an atomic level update as well, is there a way I can update a
table without providing explicit variables (aka expression) in the WHERE
clause.
e.g.: I want to update one row in a table and in the where clause i wish to
use a SELECT with other table to identify / match the record.
Kindly drop a pointer to the syntax manual to look it up.
Happy weekend !
--
Devopam Mittra
Life and Relations are not binary
Re: Hive UPDATE : query around usage
Posted by Devopam Mittra <de...@gmail.com>.
hi Lefty,
Thanks again for the crisp response.
After going through it all over again , and brooding over the weekend I
understood that my ask is yet not there and I was trying to do things in a
(yet) impractical way.
I am sure it will be the need in future , if we were to use and promote
Hive as The 'SQL' engine for Hadoop in the longer run.
Till then I will rather stick to my old technique of handling UPDATE as
DELETE+INSERT and leveraging dynamic partitions to achieve the objective.
warm regards
Devopam
On Sat, Jan 17, 2015 at 1:48 PM, Lefty Leverenz <le...@gmail.com>
wrote:
> I can't answer your question, but here are some links to documentation in
> the Hive wiki:
>
> - DML -- Update
> <https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-Update>
> - Hive Transactions
> <https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions>
>
>
> -- Lefty Leverenz
>
>
> On Fri, Jan 16, 2015 at 6:09 AM, Devopam Mittra <de...@gmail.com> wrote:
>
>> hi All,
>> I wanted to UPDATE multiple rows in a table by joining it with another
>> table for identification of matching records (otherwise in the RDBMS world
>> also commonly known as SET based UPDATE).
>>
>> The idea is to update multiple records in one go , to downsize the lead
>> time for each MR job to kick off.
>>
>> Is it supported or I am being greedy here with my wishlist ?
>>
>> Also, at an atomic level update as well, is there a way I can update a
>> table without providing explicit variables (aka expression) in the WHERE
>> clause.
>> e.g.: I want to update one row in a table and in the where clause i wish
>> to use a SELECT with other table to identify / match the record.
>>
>> Kindly drop a pointer to the syntax manual to look it up.
>>
>> Happy weekend !
>> --
>> Devopam Mittra
>> Life and Relations are not binary
>>
>
>
--
Devopam Mittra
Life and Relations are not binary
Re: Hive UPDATE : query around usage
Posted by Lefty Leverenz <le...@gmail.com>.
I can't answer your question, but here are some links to documentation in
the Hive wiki:
- DML -- Update
<https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-Update>
- Hive Transactions
<https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions>
-- Lefty Leverenz
On Fri, Jan 16, 2015 at 6:09 AM, Devopam Mittra <de...@gmail.com> wrote:
> hi All,
> I wanted to UPDATE multiple rows in a table by joining it with another
> table for identification of matching records (otherwise in the RDBMS world
> also commonly known as SET based UPDATE).
>
> The idea is to update multiple records in one go , to downsize the lead
> time for each MR job to kick off.
>
> Is it supported or I am being greedy here with my wishlist ?
>
> Also, at an atomic level update as well, is there a way I can update a
> table without providing explicit variables (aka expression) in the WHERE
> clause.
> e.g.: I want to update one row in a table and in the where clause i wish
> to use a SELECT with other table to identify / match the record.
>
> Kindly drop a pointer to the syntax manual to look it up.
>
> Happy weekend !
> --
> Devopam Mittra
> Life and Relations are not binary
>
Re: Writing Hive Query Output to local system
Posted by Sanjay Subramanian <sa...@yahoo.com>.
I am using Cloudera Manager and I know how to add a param thru Safety Valve I don't know how to ignore a param thru Safety Valve :-)
Actually I will put the question on the CDH groups as well...
Any thoughts guys ?
thanks
sanjay
From: "grimaldi.vincenzo@gmail.com" <gr...@gmail.com>
To: user@hive.apache.org; Sanjay Subramanian <sa...@yahoo.com>
Sent: Friday, January 16, 2015 11:27 AM
Subject: Re: Writing Hive Query Output to local system
In your hive-site.xml remove the <property> block corresponding to the parametereç
hive.metastore.local
That's hit. Is a deprecated parameter and is not necessary anymore.
Regards,
| | Vincenzo Grimaldi
Senior Business Intelligence Consultant p:00 39 380 52 22 218 | m:00 353 851 69 84 58 | e:grimaldi.vincenzo@gmail.com | a: Aprt. 148A, Smithfield Market, Smithfield, Dublin 7, Ireland |
Get a signature like this: Click here!
2015-01-16 19:05 GMT+00:00 Sanjay Subramanian <sa...@yahoo.com>:
hey guys
I recall this did not happen in the days of 0.9.x version
But I use 0.13.x now and when I run a hive query
hive -e "select * from tablename" > ./myfile.txt
The first line in myfile.txt is as follows
2015-01-16 10:48:13,091 WARN [main] conf.HiveConf (HiveConf.java:initialize(1491)) - DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.
How do I avoid this ?
thanks
sanjay
--
| | Vincenzo Grimaldi
Senior Business Intelligence Consultant p:00 39 380 52 22 218 | m:00 353 851 69 84 58 | e:grimaldi.vincenzo@gmail.com | a: Aprt. 148A, Smithfield Market, Smithfield, Dublin 7, Ireland |
Re: Writing Hive Query Output to local system
Posted by "grimaldi.vincenzo@gmail.com" <gr...@gmail.com>.
In your hive-site.xml remove the <property> block corresponding to the
parametereç
hive.metastore.local
That's hit. Is a deprecated parameter and is not necessary anymore.
Regards,
[image: photo]
*Vincenzo Grimaldi*
Senior Business Intelligence Consultant
p:00 39 380 52 22 218 | m:00 353 851 69 84 58 | e:
grimaldi.vincenzo@gmail.com | a: Aprt. 148A, Smithfield Market, Smithfield,
Dublin 7, Ireland
<http://facebook.com/vgrimaldi2>
<http://ie.linkedin.com/pub/vincenzo-grimaldi/14/422/bb1/>
Get a signature like this:
<http://ws-stats.appspot.com/r?rdata=eyJydXJsIjogImh0dHA6Ly93d3cud2lzZXN0YW1wLmNvbS8/dXRtX3NvdXJjZT1leHRlbnNpb24mdXRtX21lZGl1bT1lbWFpbCZ1dG1fY2FtcGFpZ249cHJvbW9fNDUiLCAiZSI6ICJwcm9tb180NV9jbGljayJ9>
Click
here!
<http://ws-stats.appspot.com/r?rdata=eyJydXJsIjogImh0dHA6Ly93d3cud2lzZXN0YW1wLmNvbS8/dXRtX3NvdXJjZT1leHRlbnNpb24mdXRtX21lZGl1bT1lbWFpbCZ1dG1fY2FtcGFpZ249cHJvbW9fNDUiLCAiZSI6ICJwcm9tb180NV9jbGljayJ9>
2015-01-16 19:05 GMT+00:00 Sanjay Subramanian <sa...@yahoo.com>:
> hey guys
>
> I recall this did not happen in the days of 0.9.x version
>
> But I use 0.13.x now and when I run a hive query
>
> hive -e "select * from tablename" > ./myfile.txt
>
> The first line in myfile.txt is as follows
>
> 2015-01-16 10:48:13,091 WARN [main] conf.HiveConf
> (HiveConf.java:initialize(1491)) - DEPRECATED: Configuration property
> hive.metastore.local no longer has any effect. Make sure to provide a valid
> value for hive.metastore.uris if you are connecting to a remote metastore.
>
> How do I avoid this ?
>
> thanks
>
> sanjay
>
--
[image: photo]
*Vincenzo Grimaldi*
Senior Business Intelligence Consultant
p:00 39 380 52 22 218 | m:00 353 851 69 84 58 | e:
grimaldi.vincenzo@gmail.com | a: Aprt. 148A, Smithfield Market, Smithfield,
Dublin 7, Ireland
<https://mail.google.com/mail/u/0/facebook.com/vgrimaldi2>
<https://mail.google.com/mail/u/0/ie.linkedin.com/pub/vincenzo-grimaldi/14/422/bb1/>
Writing Hive Query Output to local system
Posted by Sanjay Subramanian <sa...@yahoo.com>.
hey guys
I recall this did not happen in the days of 0.9.x version
But I use 0.13.x now and when I run a hive query
hive -e "select * from tablename" > ./myfile.txt
The first line in myfile.txt is as follows
2015-01-16 10:48:13,091 WARN [main] conf.HiveConf (HiveConf.java:initialize(1491)) - DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.
How do I avoid this ?
thanks
sanjay