You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Guy Doulberg <Gu...@perion.com> on 2015/02/02 16:51:40 UTC

UDF using third party jar 

Hi

I am migrating my hive from CDH4 to CDH5(1.2),

I had a udf that extracts the useragent values out of the useragent string,

I am using the third-party uadetector library,

This UDF worked well in the CDH4

After the migration it stopped working because of:

Exception running child : java.lang.RuntimeException: java.lang.NoSuchMethodError: net.sf.uadetector.UserAgentStringParser.parse(Ljava/lang/String;)Lnet/sf/uadetector/ReadableUserAgent;
        at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:175)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1554)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)


In my code I am referring to the parse method of UserAgentStringParser, I add the jars required.

I also tried to override getRequiredJars, but the issue remains

Any idea?

Thanks,
Guy Doulberg

Re: UDF using third party jar

Posted by Lukas Nalezenec <lu...@firma.seznam.cz>.
Hi,
it looks like jar version conflict - you might have two versions of jar 
containing class "net.sf.uadetector.UserAgentStringParser" on your 
classpath.
Lukas

On 2.2.2015 16:51, Guy Doulberg wrote:
>
> Hi
>
> I am migrating my hive from CDH4 to CDH5(1.2),
>
> I had a udf that extracts the useragent values out of the useragent 
> string,
>
> I am using the third-party uadetector library,
>
> This UDF worked well in the CDH4
>
> After the migration it stopped working because of:
>
> Exception running child : java.lang.RuntimeException: 
> java.lang.NoSuchMethodError: 
> net.sf.uadetector.UserAgentStringParser.parse(Ljava/lang/String;)Lnet/sf/uadetector/ReadableUserAgent;
>
>         at 
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:175)
>
>         at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
>
>         at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
>
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
>
>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
>
>         at java.security.AccessController.doPrivileged(Native Method)
>
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>
>         at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1554)
>
>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
>
> In my code I am referring to the parse method of 
> UserAgentStringParser, I add the jars required.
>
> I also tried to override getRequiredJars, but the issue remains
>
> Any idea?
>
> Thanks,
>
> Guy Doulberg
>