You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Ayon Sinha <ay...@yahoo.com> on 2011/08/03 05:39:16 UTC

How to install UDF jar into Hive server

Hi,
I have UDF jar that I need to install so that the Hive web clients can execute that function. Can someone point me to the doc that describes it?
 
-Ayon
See My Photos on Flickr
Also check out my Blog for answers to commonly asked questions.

Re: How to install UDF jar into Hive server

Posted by Ayon Sinha <ay...@yahoo.com>.
Hi Edward,
I think option 1 with hive.rc is the path we want to take.
I am really not looking for web-client users ton install UDF jars. Our use-case is:
We as the Infrastructure team have written a UDF that we want our internal web-client customers to use without them having to install any jars. They wouldn't know the difference in usage between 
to_date(string) vs our_custom_fn(string).

Let me know if you have more detailed steps for option 1.
 
-Ayon
See My Photos on Flickr
Also check out my Blog for answers to commonly asked questions.



________________________________
From: Edward Capriolo <ed...@gmail.com>
To: user@hive.apache.org; Ayon Sinha <ay...@yahoo.com>
Cc: "chinnarao@huawei.com" <ch...@huawei.com>
Sent: Wednesday, August 3, 2011 7:31 AM
Subject: Re: How to install UDF jar into Hive server





On Wed, Aug 3, 2011 at 2:55 AM, Ayon Sinha <ay...@yahoo.com> wrote:

So if I have a web-client calling the Hive server, will step 1 & 2 need to be executed for every call as a sequence (that would be inconvenient). I really want this UDF to be available to every user of our web interface. They shouldn't need to know where the jar is.
> 
>-Ayon
>See My Photos on Flickr
>Also check out my Blog for answers to commonly asked questions.
>
>
>
>________________________________
>From: Chinna <ch...@huawei.com>
>To: user@hive.apache.org; 'Ayon Sinha' <ay...@yahoo.com>
>Sent: Tuesday, August 2, 2011 11:38 PM
>Subject: RE: How to install UDF jar into Hive server
>
>
>Hi,
>   
> For example u have an UDF jar
like   lower.jar (Here u have the implementation for the lower
function) in /home/myjars location.
> 
>if it is Hive CLI or Hive Server or web
client... what mode u r using there u need to execute the following queries
> 
>1. You need to add the jar  to Hive
class path for this u can execute this query
> 
> add jar '/home/myjars/lower.jar';
> 
>2. U need to create the function using
this query
> 
> create temporary fucntion 
my_lower as 'com.example.hive.udf.Lower';
> 
>then u r function is ready now u can use the
function like
> 
>select my_lower(title) from titles
> 
>-Chinna Rao Lalam
>
>________________________________
> 
>From:Ayon Sinha
[mailto:ayonsinha@yahoo.com] 
>Sent: Wednesday, August 03, 2011
10:50 AM
>To: user@hive.apache.org
>Subject: Re: How to install UDF
jar into Hive server
> 
>Thanks
Chinna. Unfortunately this doesn't say how to install it such that the
HiveServer is able to use this jar. Examples in this wiki link does it all in
the Hive CLI.
> 
>-Ayon
>See My Photos
on Flickr
>Also check out my
Blog for answers to commonly asked questions.
>
>________________________________
> 
>From:Chinna
<ch...@huawei.com>
>To: user@hive.apache.org; 'Ayon
Sinha' <ay...@yahoo.com>
>Sent: Tuesday, August 2, 2011 9:22
PM
>Subject: RE: How to install UDF
jar into Hive server
>Hi,
>  
>  In the below link
this section "creating custom UDFs"  will help u ...
> 
>  https://cwiki.apache.org/confluence/display/Hive/HivePlugins
> 
>- Chinna Rao
Lalam
>
>________________________________
> 
>From:Ayon Sinha
[mailto:ayonsinha@yahoo.com] 
>Sent: Wednesday, August 03, 2011
9:09 AM
>To: Hive Mailinglist
>Subject: How to install UDF jar
into Hive server
> 
>Hi,
>I have
UDF jar that I need to install so that the Hive web clients can execute that
function. Can someone point me to the doc that describes it?
> 
>-Ayon
>See My Photos
on Flickr
>Also check out my Blog
for answers to commonly asked questions.
> 
>
>

Users have no way to send the hive-service jars but you can work around it.

1) Option 1: hive now has a hive.rc file. You can put this on the hive service server.
2) Option 2: There was a ticket open to add jars that are present on hdfs. I believe it was completed. 
3) Option 3: (I think this works) 
(from client)
hadoo dfs -copyFromLocal myjar.jar /tmp;
(inside hive-service session)
dfs -copyToLocal /tmp/myjar.jar /tmp ;
add jar /tmp/my.jar;

Edward

Re: How to install UDF jar into Hive server

Posted by Edward Capriolo <ed...@gmail.com>.
On Wed, Aug 3, 2011 at 2:55 AM, Ayon Sinha <ay...@yahoo.com> wrote:

> So if I have a web-client calling the Hive server, will step 1 & 2 need to
> be executed for every call as a sequence (that would be inconvenient). I
> really want this UDF to be available to every user of our web interface.
> They shouldn't need to know where the jar is.
>
> -Ayon
> See My Photos on Flickr <http://www.flickr.com/photos/ayonsinha/>
> Also check out my Blog for answers to commonly asked questions.<http://dailyadvisor.blogspot.com>
>
> ------------------------------
> *From:* Chinna <ch...@huawei.com>
> *To:* user@hive.apache.org; 'Ayon Sinha' <ay...@yahoo.com>
> *Sent:* Tuesday, August 2, 2011 11:38 PM
> *Subject:* RE: How to install UDF jar into Hive server
>
>  Hi,
>
>  For example u have an UDF jar like   lower.jar (Here u have the
> implementation for the lower function) in /home/myjars location.
>
> if it is Hive CLI or Hive Server or web client... what mode u r using there
> u need to execute the following queries
>
> 1. You need to add the jar  to Hive class path for this u can execute this
> query
>
>  add jar '/home/myjars/lower.jar';
>
> 2. U need to create the function using this query
>
>  create temporary fucntion  my_lower as 'com.example.hive.udf.Lower';
>
> then u r function is ready now u can use the function like
>
> select my_lower(title) from titles
>
>  -Chinna Rao Lalam
>   ------------------------------
>  *From:* Ayon Sinha [mailto:ayonsinha@yahoo.com]
> *Sent:* Wednesday, August 03, 2011 10:50 AM
> *To:* user@hive.apache.org
> *Subject:* Re: How to install UDF jar into Hive server
>
>  Thanks Chinna. Unfortunately this doesn't say how to install it such that
> the HiveServer is able to use this jar. Examples in this wiki link does it
> all in the Hive CLI.
>
>  -Ayon
> See My Photos on Flickr <http://www.flickr.com/photos/ayonsinha/>
> Also check out my Blog for answers to commonly asked questions.<http://dailyadvisor.blogspot.com>
>   ------------------------------
>  *From:* Chinna <ch...@huawei.com>
> *To:* user@hive.apache.org; 'Ayon Sinha' <ay...@yahoo.com>
> *Sent:* Tuesday, August 2, 2011 9:22 PM
> *Subject:* RE: How to install UDF jar into Hive server
>   Hi,
>
>    In the below link this section "* **creating custom UDFs*"  will help u
> ...
>
>    https://cwiki.apache.org/confluence/display/Hive/HivePlugins
>
>   - Chinna Rao Lalam
>   ------------------------------
>  *From:* Ayon Sinha [mailto:ayonsinha@yahoo.com]
> *Sent:* Wednesday, August 03, 2011 9:09 AM
> *To:* Hive Mailinglist
> *Subject:* How to install UDF jar into Hive server
>
>   Hi,
>   I have UDF jar that I need to install so that the Hive web clients can
> execute that function. Can someone point me to the doc that describes it?
>
>   -Ayon
> See My Photos on Flickr <http://www.flickr.com/photos/ayonsinha/>
> Also check out my Blog for answers to commonly asked questions.<http://dailyadvisor.blogspot.com>
>
>
>
>
Users have no way to send the hive-service jars but you can work around it.

1) Option 1: hive now has a hive.rc file. You can put this on the hive
service server.
2) Option 2: There was a ticket open to add jars that are present on hdfs. I
believe it was completed.
3) Option 3: (I think this works)
(from client)
hadoo dfs -copyFromLocal myjar.jar /tmp;
(inside hive-service session)
dfs -copyToLocal /tmp/myjar.jar /tmp ;
add jar /tmp/my.jar;

Edward

Re: How to install UDF jar into Hive server

Posted by Ayon Sinha <ay...@yahoo.com>.
So if I have a web-client calling the Hive server, will step 1 & 2 need to be executed for every call as a sequence (that would be inconvenient). I really want this UDF to be available to every user of our web interface. They shouldn't need to know where the jar is.
 
-Ayon
See My Photos on Flickr
Also check out my Blog for answers to commonly asked questions.



________________________________
From: Chinna <ch...@huawei.com>
To: user@hive.apache.org; 'Ayon Sinha' <ay...@yahoo.com>
Sent: Tuesday, August 2, 2011 11:38 PM
Subject: RE: How to install UDF jar into Hive server


 
Hi,
   
 For example u have an UDF jar
like   lower.jar (Here u have the implementation for the lower
function) in /home/myjars location.
 
if it is Hive CLI or Hive Server or web
client... what mode u r using there u need to execute the following queries
 
1. You need to add the jar  to Hive
class path for this u can execute this query
 
 add jar '/home/myjars/lower.jar';
 
2. U need to create the function using
this query
 
 create temporary fucntion 
my_lower as 'com.example.hive.udf.Lower';
 
then u r function is ready now u can use the
function like
 
select my_lower(title) from titles
 
-Chinna Rao Lalam

________________________________
 
From:Ayon Sinha
[mailto:ayonsinha@yahoo.com] 
Sent: Wednesday, August 03, 2011
10:50 AM
To: user@hive.apache.org
Subject: Re: How to install UDF
jar into Hive server
 
Thanks
Chinna. Unfortunately this doesn't say how to install it such that the
HiveServer is able to use this jar. Examples in this wiki link does it all in
the Hive CLI.
 
-Ayon
See My Photos
on Flickr
Also check out my
Blog for answers to commonly asked questions.

________________________________
 
From:Chinna
<ch...@huawei.com>
To: user@hive.apache.org; 'Ayon
Sinha' <ay...@yahoo.com>
Sent: Tuesday, August 2, 2011 9:22
PM
Subject: RE: How to install UDF
jar into Hive server
Hi,
  
  In the below link
this section "creating custom UDFs"  will help u ...
 
 
https://cwiki.apache.org/confluence/display/Hive/HivePlugins
 
- Chinna Rao
Lalam

________________________________
 
From:Ayon Sinha
[mailto:ayonsinha@yahoo.com] 
Sent: Wednesday, August 03, 2011
9:09 AM
To: Hive Mailinglist
Subject: How to install UDF jar
into Hive server
 
Hi,
I have
UDF jar that I need to install so that the Hive web clients can execute that
function. Can someone point me to the doc that describes it?
 
-Ayon
See My Photos
on Flickr
Also check out my Blog
for answers to commonly asked questions.

RE: How to install UDF jar into Hive server

Posted by Chinna <ch...@huawei.com>.
Hi,

   

 For example u have an UDF jar like   lower.jar (Here u have the
implementation for the lower function) in /home/myjars location.

 

if it is Hive CLI or Hive Server or web client... what mode u r using there
u need to execute the following queries

 

1. You need to add the jar  to Hive class path for this u can execute this
query

 

 add jar '/home/myjars/lower.jar';

 

2. U need to create the function using this query

 

 create temporary fucntion  my_lower as 'com.example.hive.udf.Lower';

 

then u r function is ready now u can use the function like

 

select my_lower(title) from titles

 

-Chinna Rao Lalam

  _____  

From: Ayon Sinha [mailto:ayonsinha@yahoo.com] 
Sent: Wednesday, August 03, 2011 10:50 AM
To: user@hive.apache.org
Subject: Re: How to install UDF jar into Hive server

 

Thanks Chinna. Unfortunately this doesn't say how to install it such that
the HiveServer is able to use this jar. Examples in this wiki link does it
all in the Hive CLI.

 

-Ayon
See My Photos <http://www.flickr.com/photos/ayonsinha/>  on Flickr
Also check out my <http://dailyadvisor.blogspot.com>  Blog for answers to
commonly asked questions.

  _____  

From: Chinna <ch...@huawei.com>
To: user@hive.apache.org; 'Ayon Sinha' <ay...@yahoo.com>
Sent: Tuesday, August 2, 2011 9:22 PM
Subject: RE: How to install UDF jar into Hive server

Hi,

  

  In the below link this section " creating custom UDFs"  will help u ...

 

  https://cwiki.apache.org/confluence/display/Hive/HivePlugins

 

- Chinna Rao Lalam

  _____  

From: Ayon Sinha [mailto:ayonsinha@yahoo.com] 
Sent: Wednesday, August 03, 2011 9:09 AM
To: Hive Mailinglist
Subject: How to install UDF jar into Hive server

 

Hi,

I have UDF jar that I need to install so that the Hive web clients can
execute that function. Can someone point me to the doc that describes it?

 

-Ayon
See My Photos <http://www.flickr.com/photos/ayonsinha/>  on Flickr
Also check out my Blog <http://dailyadvisor.blogspot.com>  for answers to
commonly asked questions.

 


Re: How to install UDF jar into Hive server

Posted by Ayon Sinha <ay...@yahoo.com>.
Thanks Chinna. Unfortunately this doesn't say how to install it such that the HiveServer is able to use this jar. Examples in this wiki link does it all in the Hive CLI.
 
-Ayon
See My Photos on Flickr
Also check out my Blog for answers to commonly asked questions.



________________________________
From: Chinna <ch...@huawei.com>
To: user@hive.apache.org; 'Ayon Sinha' <ay...@yahoo.com>
Sent: Tuesday, August 2, 2011 9:22 PM
Subject: RE: How to install UDF jar into Hive server


 
Hi,
  
  In the below link this section "creating custom
UDFs"  will help u ...
 
  https://cwiki.apache.org/confluence/display/Hive/HivePlugins
 
- Chinna Rao Lalam

________________________________
 
From:Ayon Sinha
[mailto:ayonsinha@yahoo.com] 
Sent: Wednesday, August 03, 2011
9:09 AM
To: Hive Mailinglist
Subject: How to install UDF jar
into Hive server
 
Hi,
I have
UDF jar that I need to install so that the Hive web clients can execute that
function. Can someone point me to the doc that describes it?
 
-Ayon
See My Photos
on Flickr
Also check out my
Blog for answers to commonly asked questions.

RE: How to install UDF jar into Hive server

Posted by Chinna <ch...@huawei.com>.
Hi,

  

  In the below link this section " creating custom UDFs"  will help u ...

 

  https://cwiki.apache.org/confluence/display/Hive/HivePlugins

 

- Chinna Rao Lalam

  _____  

From: Ayon Sinha [mailto:ayonsinha@yahoo.com] 
Sent: Wednesday, August 03, 2011 9:09 AM
To: Hive Mailinglist
Subject: How to install UDF jar into Hive server

 

Hi,

I have UDF jar that I need to install so that the Hive web clients can
execute that function. Can someone point me to the doc that describes it?

 

-Ayon
See My Photos <http://www.flickr.com/photos/ayonsinha/>  on Flickr
Also check out my <http://dailyadvisor.blogspot.com>  Blog for answers to
commonly asked questions.