You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Xiaoyong Zhu <xi...@microsoft.com> on 2014/12/18 10:26:21 UTC
Hive UDFs?
Hi experts, I heard that if I want to use Hive UDFs, I must deploy the jars to all the machines that are running Hive (which is painful to me..)
After reading this<https://cwiki.apache.org/confluence/display/Hive/HivePlugins> I didn't find any document talking about this - could someone help to answer the following questions:
1. Do I really need to do so(deploy UDF jars to all the machines and then use it)?
2. Is there a possibility that I could use other language (rather than java) to write those UDFs?
Thanks!
Xiaoyong
RE: Hive UDFs?
Posted by Xiaoyong Zhu <xi...@microsoft.com>.
+ Rathish
Xiaoyong
-----Original Message-----
From: Xiaoyong Zhu
Sent: Thursday, December 18, 2014 5:56 PM
To: user@hive.apache.org
Subject: RE: Hive UDFs?
Thanks!
For Q1 - is there a way that Hive helps me to automatically do this (for example, I can register the UDF somewhere and the UDF gets automatically distributed)? Or I need to login to each node to ensure this happens?
Xiaoyong
-----Original Message-----
From: Rathish A M [mailto:rathish4u@gmail.com]
Sent: Thursday, December 18, 2014 5:36 PM
To: user@hive.apache.org
Subject: Re: Hive UDFs?
Hi Xiayong,
Q 1 :- You need to deploy the jar in the server where your hive is running.
Q 2 :- Which language you are looking instead of Java ? . As per my understanding you can write UDF functions in python also ( I have not tried this from my end )
Regards,
Rathish
On Thu, Dec 18, 2014 at 2:56 PM, Xiaoyong Zhu <xi...@microsoft.com> wrote:
> Hi experts, I heard that if I want to use Hive UDFs, I must deploy the
> jars to all the machines that are running Hive (which is painful to
> me..)
>
> After reading this I didn’t find any document talking about this –
> could someone help to answer the following questions:
>
> 1. Do I really need to do so(deploy UDF jars to all the machines and
> then use it)?
>
> 2. Is there a possibility that I could use other language (rather than
> java) to write those UDFs?
>
>
>
> Thanks!
>
>
>
> Xiaoyong
>
>
--
Regards,
-------------------------
Rathish A M
Bangalore
RE: Hive UDFs?
Posted by Xiaoyong Zhu <xi...@microsoft.com>.
I see, thanks!
Xiaoyong
From: Jason Dere [mailto:jdere@hortonworks.com]
Sent: Friday, December 19, 2014 3:52 AM
To: user@hive.apache.org
Subject: Re: Hive UDFs?
Another option is https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-PermanentFunctions, as another user mentioned on this list a few days ago.
On Dec 18, 2014, at 5:54 AM, Stéphane Verlet <ka...@gmail.com>> wrote:
You only need the jar on the computer where you execute the hive command , not on all cluster nodes
On Thu, Dec 18, 2014 at 2:55 AM, Xiaoyong Zhu <xi...@microsoft.com>> wrote:
Thanks!
For Q1 - is there a way that Hive helps me to automatically do this (for example, I can register the UDF somewhere and the UDF gets automatically distributed)? Or I need to login to each node to ensure this happens?
Xiaoyong
-----Original Message-----
From: Rathish A M [mailto:rathish4u@gmail.com<ma...@gmail.com>]
Sent: Thursday, December 18, 2014 5:36 PM
To: user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: Hive UDFs?
Hi Xiayong,
Q 1 :- You need to deploy the jar in the server where your hive is running.
Q 2 :- Which language you are looking instead of Java ? . As per my understanding you can write UDF functions in python also ( I have not tried this from my end )
Regards,
Rathish
On Thu, Dec 18, 2014 at 2:56 PM, Xiaoyong Zhu <xi...@microsoft.com>> wrote:
> Hi experts, I heard that if I want to use Hive UDFs, I must deploy the
> jars to all the machines that are running Hive (which is painful to
> me..)
>
> After reading this I didn't find any document talking about this -
> could someone help to answer the following questions:
>
> 1. Do I really need to do so(deploy UDF jars to all the machines and
> then use it)?
>
> 2. Is there a possibility that I could use other language (rather than
> java) to write those UDFs?
>
>
>
> Thanks!
>
>
>
> Xiaoyong
>
>
--
Regards,
-------------------------
Rathish A M
Bangalore
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: Hive UDFs?
Posted by Jason Dere <jd...@hortonworks.com>.
Another option is https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-PermanentFunctions, as another user mentioned on this list a few days ago.
On Dec 18, 2014, at 5:54 AM, Stéphane Verlet <ka...@gmail.com> wrote:
> You only need the jar on the computer where you execute the hive command , not on all cluster nodes
>
>
>
> On Thu, Dec 18, 2014 at 2:55 AM, Xiaoyong Zhu <xi...@microsoft.com> wrote:
> Thanks!
> For Q1 - is there a way that Hive helps me to automatically do this (for example, I can register the UDF somewhere and the UDF gets automatically distributed)? Or I need to login to each node to ensure this happens?
>
> Xiaoyong
>
> -----Original Message-----
> From: Rathish A M [mailto:rathish4u@gmail.com]
> Sent: Thursday, December 18, 2014 5:36 PM
> To: user@hive.apache.org
> Subject: Re: Hive UDFs?
>
> Hi Xiayong,
>
> Q 1 :- You need to deploy the jar in the server where your hive is running.
> Q 2 :- Which language you are looking instead of Java ? . As per my understanding you can write UDF functions in python also ( I have not tried this from my end )
>
> Regards,
> Rathish
>
>
> On Thu, Dec 18, 2014 at 2:56 PM, Xiaoyong Zhu <xi...@microsoft.com> wrote:
> > Hi experts, I heard that if I want to use Hive UDFs, I must deploy the
> > jars to all the machines that are running Hive (which is painful to
> > me..)
> >
> > After reading this I didn’t find any document talking about this –
> > could someone help to answer the following questions:
> >
> > 1. Do I really need to do so(deploy UDF jars to all the machines and
> > then use it)?
> >
> > 2. Is there a possibility that I could use other language (rather than
> > java) to write those UDFs?
> >
> >
> >
> > Thanks!
> >
> >
> >
> > Xiaoyong
> >
> >
>
>
>
> --
> Regards,
> -------------------------
> Rathish A M
> Bangalore
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Hive UDFs?
Posted by Stéphane Verlet <ka...@gmail.com>.
You only need the jar on the computer where you execute the hive command ,
not on all cluster nodes
On Thu, Dec 18, 2014 at 2:55 AM, Xiaoyong Zhu <xi...@microsoft.com>
wrote:
>
> Thanks!
> For Q1 - is there a way that Hive helps me to automatically do this (for
> example, I can register the UDF somewhere and the UDF gets automatically
> distributed)? Or I need to login to each node to ensure this happens?
>
> Xiaoyong
>
> -----Original Message-----
> From: Rathish A M [mailto:rathish4u@gmail.com]
> Sent: Thursday, December 18, 2014 5:36 PM
> To: user@hive.apache.org
> Subject: Re: Hive UDFs?
>
> Hi Xiayong,
>
> Q 1 :- You need to deploy the jar in the server where your hive is running.
> Q 2 :- Which language you are looking instead of Java ? . As per my
> understanding you can write UDF functions in python also ( I have not tried
> this from my end )
>
> Regards,
> Rathish
>
>
> On Thu, Dec 18, 2014 at 2:56 PM, Xiaoyong Zhu <xi...@microsoft.com>
> wrote:
> > Hi experts, I heard that if I want to use Hive UDFs, I must deploy the
> > jars to all the machines that are running Hive (which is painful to
> > me..)
> >
> > After reading this I didn’t find any document talking about this –
> > could someone help to answer the following questions:
> >
> > 1. Do I really need to do so(deploy UDF jars to all the machines
> and
> > then use it)?
> >
> > 2. Is there a possibility that I could use other language (rather
> than
> > java) to write those UDFs?
> >
> >
> >
> > Thanks!
> >
> >
> >
> > Xiaoyong
> >
> >
>
>
>
> --
> Regards,
> -------------------------
> Rathish A M
> Bangalore
>
RE: Hive UDFs?
Posted by Xiaoyong Zhu <xi...@microsoft.com>.
Thanks!
For Q1 - is there a way that Hive helps me to automatically do this (for example, I can register the UDF somewhere and the UDF gets automatically distributed)? Or I need to login to each node to ensure this happens?
Xiaoyong
-----Original Message-----
From: Rathish A M [mailto:rathish4u@gmail.com]
Sent: Thursday, December 18, 2014 5:36 PM
To: user@hive.apache.org
Subject: Re: Hive UDFs?
Hi Xiayong,
Q 1 :- You need to deploy the jar in the server where your hive is running.
Q 2 :- Which language you are looking instead of Java ? . As per my understanding you can write UDF functions in python also ( I have not tried this from my end )
Regards,
Rathish
On Thu, Dec 18, 2014 at 2:56 PM, Xiaoyong Zhu <xi...@microsoft.com> wrote:
> Hi experts, I heard that if I want to use Hive UDFs, I must deploy the
> jars to all the machines that are running Hive (which is painful to
> me..)
>
> After reading this I didn’t find any document talking about this –
> could someone help to answer the following questions:
>
> 1. Do I really need to do so(deploy UDF jars to all the machines and
> then use it)?
>
> 2. Is there a possibility that I could use other language (rather than
> java) to write those UDFs?
>
>
>
> Thanks!
>
>
>
> Xiaoyong
>
>
--
Regards,
-------------------------
Rathish A M
Bangalore
Re: Hive UDFs?
Posted by Rathish A M <ra...@gmail.com>.
Hi Xiayong,
Q 1 :- You need to deploy the jar in the server where your hive is running.
Q 2 :- Which language you are looking instead of Java ? . As per my
understanding you can write UDF functions in python also ( I have not
tried this from my end )
Regards,
Rathish
On Thu, Dec 18, 2014 at 2:56 PM, Xiaoyong Zhu <xi...@microsoft.com> wrote:
> Hi experts, I heard that if I want to use Hive UDFs, I must deploy the jars
> to all the machines that are running Hive (which is painful to me..)
>
> After reading this I didn’t find any document talking about this – could
> someone help to answer the following questions:
>
> 1. Do I really need to do so(deploy UDF jars to all the machines and
> then use it)?
>
> 2. Is there a possibility that I could use other language (rather than
> java) to write those UDFs?
>
>
>
> Thanks!
>
>
>
> Xiaoyong
>
>
--
Regards,
-------------------------
Rathish A M
Bangalore