native code in UDF

[Sean Timm <ti...@aol.com>]

I have a UDF which I'd like to use native code in via JNI.  Is there a 
way in pig that I can distributed the shared object libraries to the 
nodes?  I could manually push the .so's to the nodes or possibly use 
something like JDotSoft's JarClassLoader, but I was hoping that there 
was an easier way.

Thanks,
Sean

Re: native code in UDF

[Sean Timm <ti...@aol.com>]

Thanks!  That worked.

-Sean

Mridul Muralidharan wrote:
>
> I have tried both the distributed cache approach of hadoop and 
> manually copying off hdfs (when distributed cache was not available) 
> and doing a System.loadLibrary with the absolute path of the library.
>
>
> Both work for udf's (do it within an init() which checks against some 
> transient variable to get around lack of lifecycle methods).
>
> Regards,
> Mridul
>
>
> Sean Timm wrote:
>> I have a UDF which I'd like to use native code in via JNI.  Is there 
>> a way in pig that I can distributed the shared object libraries to 
>> the nodes?  I could manually push the .so's to the nodes or possibly 
>> use something like JDotSoft's JarClassLoader, but I was hoping that 
>> there was an easier way.
>>
>> Thanks,
>> Sean
>

Re: native code in UDF

[Mridul Muralidharan <mr...@yahoo-inc.com>]

I have tried both the distributed cache approach of hadoop and manually 
copying off hdfs (when distributed cache was not available) and doing a 
System.loadLibrary with the absolute path of the library.


Both work for udf's (do it within an init() which checks against some 
transient variable to get around lack of lifecycle methods).

Regards,
Mridul


Sean Timm wrote:
> I have a UDF which I'd like to use native code in via JNI.  Is there a 
> way in pig that I can distributed the shared object libraries to the 
> nodes?  I could manually push the .so's to the nodes or possibly use 
> something like JDotSoft's JarClassLoader, but I was hoping that there 
> was an easier way.
> 
> Thanks,
> Sean