Passing mapreduce configuration parameters to hive udf

[Jon Bender <jo...@gmail.com>]

Hi there,

I'm trying to pass some external properties to a UDF.  In the MapReduce
world I'm used to extending Configured in my classes, but in my UDF class
when initializing a new Configuration object or HiveConf object it doesn't
inherit any of those properties.  I see it in the Job Configuration XML
when the job runs but my UDF can't pick it up when it creates a new
instance.

Are there any other suggested ways of doing this?  I could probably just
add some conf file to distributed cache and load the properties on UDF
initialization, but I figured I could get at the configuration through
other means.

Thanks in advance,
Jon

Re: Passing mapreduce configuration parameters to hive udf

[Abdelrahman Shettia <as...@hortonworks.com>]

Hi Jon,

Please refer to the following document:

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Select#LanguageManualSelect-REGEXColumnSpecification

Hope this helps.

Thanks
-Abdelrahman


On Tue, Aug 13, 2013 at 9:13 AM, Jon Bender <jo...@gmail.com>wrote:

> Hi there,
>
> I'm trying to pass some external properties to a UDF.  In the MapReduce
> world I'm used to extending Configured in my classes, but in my UDF class
> when initializing a new Configuration object or HiveConf object it doesn't
> inherit any of those properties.  I see it in the Job Configuration XML
> when the job runs but my UDF can't pick it up when it creates a new
> instance.
>
> Are there any other suggested ways of doing this?  I could probably just
> add some conf file to distributed cache and load the properties on UDF
> initialization, but I figured I could get at the configuration through
> other means.
>
> Thanks in advance,
> Jon
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.