You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by tom kersnick <hi...@gmail.com> on 2010/03/11 01:55:42 UTC
Hive UDF Unknown exception:
Gents,
Any ideas why this happens? Im using hive 0.50 with hadoop 20.2.
This is a super simple UDF.....
Im just taking the length of the values and then dividing by pi. It keeps
popping up with this error:
FAILED: Unknown exception: [D cannot be cast to [Ljava.lang.Object;
Here is my approach:
package com.xyz.udf;
import org.apache.hadoop.hive.ql.exec.UDF;
import java.util.Collections;
public final class test extends UDF {
public double evaluate(double[] values) {
final Integer len = values.length;
final Integer pi = len / 3.14159265;
return values[pi];
}
}
hive> list
jars;
hive> add jar /tmp/hive_aux/x-y-z-udf-1.0-SNAPSHOT.jar;
Added /tmp/hive_aux/x-y-z-udf-1.0-SNAPSHOT.jar to class path
hive> create temporary function my_test as 'com.xyz.udf.test';
OK
Time taken: 0.41 seconds
hive> show
tables;
OK
userpool
test
Time taken: 3.167 seconds
hive> describe userpool;
OK
word string
amount int
Time taken: 0.098 seconds
hive> select my_test(amount) from
userpool;
FAILED: Unknown exception: [D cannot be cast to [Ljava.lang.Object;
hive> describe test;
OK
word string
amount string
Time taken: 0.134 seconds
hive> select my_test(amount) from
test;
FAILED: Unknown exception: [D cannot be cast to [Ljava.lang.Object;
Thanks in advance!
/tom
Re: Hive UDF Unknown exception:
Posted by tom kersnick <hi...@gmail.com>.
That worked!
Thanks a ton!
On Wed, Mar 10, 2010 at 5:56 PM, Mafish Liu <ma...@gmail.com> wrote:
> 2010/3/11 tom kersnick <hi...@gmail.com>:
> > Tried this:
> >
> > package com.eharmony.pipeline.analytics.udf;
> >
> > import org.apache.hadoop.hive.ql.exec.UDF;
> > import java.util.Collections;
> >
> >
> > public final class test extends UDF {
> > public double evaluate("Double[]" values) {
> No " in your code and also try return Double instead of double.
> > final Integer len = values.length;
> > final Integer half = len / 2;
> > return values[half];
> > }
> > }
> >
> > Now it will not compile using Maven:
> >
> > test.java:[8,27] illegal start of type
> >
> > test.java:[8,37] ')' expected
> >
> > test.java:[8,44] ';' expected
> >
> > Thanks in advance.
> >
> > /tom
> >
> >
> >
> > On Wed, Mar 10, 2010 at 5:06 PM, Zheng Shao <zs...@gmail.com> wrote:
> >>
> >> Try "Double[]". Primitive arrays (like double[], int[]) are not
> >> supported yet, because that needs special handling for each of the
> >> primitive type.
> >>
> >> Zheng
> >>
> >> On Wed, Mar 10, 2010 at 4:55 PM, tom kersnick <hi...@gmail.com>
> wrote:
> >> > Gents,
> >> >
> >> > Any ideas why this happens? Im using hive 0.50 with hadoop 20.2.
> >> >
> >> > This is a super simple UDF.....
> >> >
> >> > Im just taking the length of the values and then dividing by pi. It
> >> > keeps
> >> > popping up with this error:
> >> >
> >> > FAILED: Unknown exception: [D cannot be cast to [Ljava.lang.Object;
> >> >
> >> > Here is my approach:
> >> >
> >> >
> >> > package com.xyz.udf;
> >> >
> >> > import org.apache.hadoop.hive.ql.exec.UDF;
> >> > import java.util.Collections;
> >> >
> >> >
> >> > public final class test extends UDF {
> >> > public double evaluate(double[] values) {
> >> > final Integer len = values.length;
> >> > final Integer pi = len / 3.14159265;
> >> > return values[pi];
> >> > }
> >> > }
> >> >
> >> >
> >> > hive> list
> >> > jars;
> >> > hive> add jar /tmp/hive_aux/x-y-z-udf-1.0-SNAPSHOT.jar;
> >> > Added /tmp/hive_aux/x-y-z-udf-1.0-SNAPSHOT.jar to class path
> >> > hive> create temporary function my_test as 'com.xyz.udf.test';
> >> > OK
> >> > Time taken: 0.41 seconds
> >> > hive> show
> >> > tables;
> >> > OK
> >> > userpool
> >> > test
> >> > Time taken: 3.167 seconds
> >> > hive> describe userpool;
> >> > OK
> >> > word string
> >> > amount int
> >> > Time taken: 0.098 seconds
> >> > hive> select my_test(amount) from
> >> > userpool;
> >> > FAILED: Unknown exception: [D cannot be cast to [Ljava.lang.Object;
> >> > hive> describe test;
> >> > OK
> >> > word string
> >> > amount string
> >> > Time taken: 0.134 seconds
> >> > hive> select my_test(amount) from
> >> > test;
> >> > FAILED: Unknown exception: [D cannot be cast to [Ljava.lang.Object;
> >> >
> >> >
> >> > Thanks in advance!
> >> >
> >> > /tom
> >> >
> >>
> >>
> >>
> >> --
> >> Yours,
> >> Zheng
> >
> >
>
>
>
> --
> Mafish@gmail.com
>
Re: Hive UDF Unknown exception:
Posted by Mafish Liu <ma...@gmail.com>.
2010/3/11 tom kersnick <hi...@gmail.com>:
> Tried this:
>
> package com.eharmony.pipeline.analytics.udf;
>
> import org.apache.hadoop.hive.ql.exec.UDF;
> import java.util.Collections;
>
>
> public final class test extends UDF {
> public double evaluate("Double[]" values) {
No " in your code and also try return Double instead of double.
> final Integer len = values.length;
> final Integer half = len / 2;
> return values[half];
> }
> }
>
> Now it will not compile using Maven:
>
> test.java:[8,27] illegal start of type
>
> test.java:[8,37] ')' expected
>
> test.java:[8,44] ';' expected
>
> Thanks in advance.
>
> /tom
>
>
>
> On Wed, Mar 10, 2010 at 5:06 PM, Zheng Shao <zs...@gmail.com> wrote:
>>
>> Try "Double[]". Primitive arrays (like double[], int[]) are not
>> supported yet, because that needs special handling for each of the
>> primitive type.
>>
>> Zheng
>>
>> On Wed, Mar 10, 2010 at 4:55 PM, tom kersnick <hi...@gmail.com> wrote:
>> > Gents,
>> >
>> > Any ideas why this happens? Im using hive 0.50 with hadoop 20.2.
>> >
>> > This is a super simple UDF.....
>> >
>> > Im just taking the length of the values and then dividing by pi. It
>> > keeps
>> > popping up with this error:
>> >
>> > FAILED: Unknown exception: [D cannot be cast to [Ljava.lang.Object;
>> >
>> > Here is my approach:
>> >
>> >
>> > package com.xyz.udf;
>> >
>> > import org.apache.hadoop.hive.ql.exec.UDF;
>> > import java.util.Collections;
>> >
>> >
>> > public final class test extends UDF {
>> > public double evaluate(double[] values) {
>> > final Integer len = values.length;
>> > final Integer pi = len / 3.14159265;
>> > return values[pi];
>> > }
>> > }
>> >
>> >
>> > hive> list
>> > jars;
>> > hive> add jar /tmp/hive_aux/x-y-z-udf-1.0-SNAPSHOT.jar;
>> > Added /tmp/hive_aux/x-y-z-udf-1.0-SNAPSHOT.jar to class path
>> > hive> create temporary function my_test as 'com.xyz.udf.test';
>> > OK
>> > Time taken: 0.41 seconds
>> > hive> show
>> > tables;
>> > OK
>> > userpool
>> > test
>> > Time taken: 3.167 seconds
>> > hive> describe userpool;
>> > OK
>> > word string
>> > amount int
>> > Time taken: 0.098 seconds
>> > hive> select my_test(amount) from
>> > userpool;
>> > FAILED: Unknown exception: [D cannot be cast to [Ljava.lang.Object;
>> > hive> describe test;
>> > OK
>> > word string
>> > amount string
>> > Time taken: 0.134 seconds
>> > hive> select my_test(amount) from
>> > test;
>> > FAILED: Unknown exception: [D cannot be cast to [Ljava.lang.Object;
>> >
>> >
>> > Thanks in advance!
>> >
>> > /tom
>> >
>>
>>
>>
>> --
>> Yours,
>> Zheng
>
>
--
Mafish@gmail.com
Re: Hive UDF Unknown exception:
Posted by tom kersnick <hi...@gmail.com>.
Tried this:
package com.eharmony.pipeline.analytics.udf;
import org.apache.hadoop.hive.ql.exec.UDF;
import java.util.Collections;
public final class test extends UDF {
public double evaluate("Double[]" values) {
final Integer len = values.length;
final Integer half = len / 2;
return values[half];
}
}
Now it will not compile using Maven:
test.java:[8,27] illegal start of type
test.java:[8,37] ')' expected
test.java:[8,44] ';' expected
Thanks in advance.
/tom
On Wed, Mar 10, 2010 at 5:06 PM, Zheng Shao <zs...@gmail.com> wrote:
> Try "Double[]". Primitive arrays (like double[], int[]) are not
> supported yet, because that needs special handling for each of the
> primitive type.
>
> Zheng
>
> On Wed, Mar 10, 2010 at 4:55 PM, tom kersnick <hi...@gmail.com> wrote:
> > Gents,
> >
> > Any ideas why this happens? Im using hive 0.50 with hadoop 20.2.
> >
> > This is a super simple UDF.....
> >
> > Im just taking the length of the values and then dividing by pi. It
> keeps
> > popping up with this error:
> >
> > FAILED: Unknown exception: [D cannot be cast to [Ljava.lang.Object;
> >
> > Here is my approach:
> >
> >
> > package com.xyz.udf;
> >
> > import org.apache.hadoop.hive.ql.exec.UDF;
> > import java.util.Collections;
> >
> >
> > public final class test extends UDF {
> > public double evaluate(double[] values) {
> > final Integer len = values.length;
> > final Integer pi = len / 3.14159265;
> > return values[pi];
> > }
> > }
> >
> >
> > hive> list
> > jars;
> > hive> add jar /tmp/hive_aux/x-y-z-udf-1.0-SNAPSHOT.jar;
> > Added /tmp/hive_aux/x-y-z-udf-1.0-SNAPSHOT.jar to class path
> > hive> create temporary function my_test as 'com.xyz.udf.test';
> > OK
> > Time taken: 0.41 seconds
> > hive> show
> > tables;
> > OK
> > userpool
> > test
> > Time taken: 3.167 seconds
> > hive> describe userpool;
> > OK
> > word string
> > amount int
> > Time taken: 0.098 seconds
> > hive> select my_test(amount) from
> > userpool;
> > FAILED: Unknown exception: [D cannot be cast to [Ljava.lang.Object;
> > hive> describe test;
> > OK
> > word string
> > amount string
> > Time taken: 0.134 seconds
> > hive> select my_test(amount) from
> > test;
> > FAILED: Unknown exception: [D cannot be cast to [Ljava.lang.Object;
> >
> >
> > Thanks in advance!
> >
> > /tom
> >
>
>
>
> --
> Yours,
> Zheng
>
Re: Hive UDF Unknown exception:
Posted by Zheng Shao <zs...@gmail.com>.
Try "Double[]". Primitive arrays (like double[], int[]) are not
supported yet, because that needs special handling for each of the
primitive type.
Zheng
On Wed, Mar 10, 2010 at 4:55 PM, tom kersnick <hi...@gmail.com> wrote:
> Gents,
>
> Any ideas why this happens? Im using hive 0.50 with hadoop 20.2.
>
> This is a super simple UDF.....
>
> Im just taking the length of the values and then dividing by pi. It keeps
> popping up with this error:
>
> FAILED: Unknown exception: [D cannot be cast to [Ljava.lang.Object;
>
> Here is my approach:
>
>
> package com.xyz.udf;
>
> import org.apache.hadoop.hive.ql.exec.UDF;
> import java.util.Collections;
>
>
> public final class test extends UDF {
> public double evaluate(double[] values) {
> final Integer len = values.length;
> final Integer pi = len / 3.14159265;
> return values[pi];
> }
> }
>
>
> hive> list
> jars;
> hive> add jar /tmp/hive_aux/x-y-z-udf-1.0-SNAPSHOT.jar;
> Added /tmp/hive_aux/x-y-z-udf-1.0-SNAPSHOT.jar to class path
> hive> create temporary function my_test as 'com.xyz.udf.test';
> OK
> Time taken: 0.41 seconds
> hive> show
> tables;
> OK
> userpool
> test
> Time taken: 3.167 seconds
> hive> describe userpool;
> OK
> word string
> amount int
> Time taken: 0.098 seconds
> hive> select my_test(amount) from
> userpool;
> FAILED: Unknown exception: [D cannot be cast to [Ljava.lang.Object;
> hive> describe test;
> OK
> word string
> amount string
> Time taken: 0.134 seconds
> hive> select my_test(amount) from
> test;
> FAILED: Unknown exception: [D cannot be cast to [Ljava.lang.Object;
>
>
> Thanks in advance!
>
> /tom
>
--
Yours,
Zheng