You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Andreas Koch <an...@gmail.com> on 2014/02/23 13:05:52 UTC

Fwd: Class not found exception from serde

I am trying to upgrade from Hive 0.10 to 0.12, but I am stuck trying to
debug an error I keep getting. I have a couple of SerDes that I have
developed, but when I try to query tables with these SerDes I get a
ClassNotFoundException for the SerDe class. I know that the jar files have
been loaded OK, because I can create tables with these SerDes, but when I
try to query the table Hive throws an exception. I know that the SerDe
interface that I use is now deprecated and I have also experimented with
extending the AbstractSerDe class in stead, but I get the same behaviour.

Any ideas?

Thanks,

Andreas


FAILED: RuntimeException org.apache.hadoop.hive.ql.metadata.HiveException:
Failed with exception java.lang.ClassNotFoundException:
my.serde.ColonSerdejava.lang.RuntimeException:
java.lang.ClassNotFoundException:
my.serde.ColonSerde
    at org.apache.hadoop.hive.ql.plan.TableDesc.getDeserializerClass(
TableDesc.java:68)
    at org.apache.hadoop.hive.ql.exec.FetchOperator.
getRowInspectorFromTable(FetchOperator.java:231)
    at org.apache.hadoop.hive.ql.exec.FetchOperator.
getOutputObjectInspector(FetchOperator.java:608)
    at org.apache.hadoop.hive.ql.exec.FetchTask.initialize(
FetchTask.java:80)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:497)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:352)
    at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:995)
    at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1038)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:921)
    at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(
CliDriver.java:268)
    at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
    at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:422)
    at org.apache.hadoop.hive.cli.CliDriver.executeDriver(
CliDriver.java:790)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:623)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.lang.ClassNotFoundException: my.serde.ColonSerde
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:190)
    at org.apache.hadoop.hive.ql.plan.TableDesc.getDeserializerClass(
TableDesc.java:66)
    ... 20 more

Re: Class not found exception from serde

Posted by Andreas Koch <an...@gmail.com>.
That is correct I added the jar file with ADD JAR.

Thanks, for the patch.


On Mon, Feb 24, 2014 at 11:44 PM, Jason Dere <jd...@hortonworks.com> wrote:

> I'm assuming the jar was loaded using ADD JAR as opposed to the jar being
> on the classpath?
>
> On Feb 24, 2014, at 2:40 PM, Jason Dere <jd...@hortonworks.com> wrote:
>
> I think TableDesc may not be using the right class loader here when it's
> trying to get the class - other parts of the code specify the class loader
> to use but in this context it does not.  I've opened
> https://issues.apache.org/jira/browse/HIVE-6495.
>
> On Feb 23, 2014, at 4:05 AM, Andreas Koch <an...@gmail.com>
> wrote:
>
>
> I am trying to upgrade from Hive 0.10 to 0.12, but I am stuck trying to
> debug an error I keep getting. I have a couple of SerDes that I have
> developed, but when I try to query tables with these SerDes I get a
> ClassNotFoundException for the SerDe class. I know that the jar files have
> been loaded OK, because I can create tables with these SerDes, but when I
> try to query the table Hive throws an exception. I know that the SerDe
> interface that I use is now deprecated and I have also experimented with
> extending the AbstractSerDe class in stead, but I get the same behaviour.
>
> Any ideas?
>
> Thanks,
>
> Andreas
>
>
> FAILED: RuntimeException org.apache.hadoop.hive.ql.metadata.HiveException:
> Failed with exception java.lang.ClassNotFoundException:
> my.serde.ColonSerdejava.lang.RuntimeException: java.lang.ClassNotFoundException:
> my.serde.ColonSerde
>     at org.apache.hadoop.hive.ql.plan.TableDesc.getDeserializerClass(
> TableDesc.java:68)
>     at org.apache.hadoop.hive.ql.exec.FetchOperator.
> getRowInspectorFromTable(FetchOperator.java:231)
>     at org.apache.hadoop.hive.ql.exec.FetchOperator.
> getOutputObjectInspector(FetchOperator.java:608)
>     at org.apache.hadoop.hive.ql.exec.FetchTask.initialize(
> FetchTask.java:80)
>     at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:497)
>     at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:352)
>     at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:995)
>     at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1038)
>     at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
>     at org.apache.hadoop.hive.ql.Driver.run(Driver.java:921)
>     at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(
> CliDriver.java:268)
>     at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
>     at org.apache.hadoop.hive.cli.CliDriver.processLine(
> CliDriver.java:422)
>     at org.apache.hadoop.hive.cli.CliDriver.executeDriver(
> CliDriver.java:790)
>     at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)
>     at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:623)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:57)
>     at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:606)
>     at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> Caused by: java.lang.ClassNotFoundException: my.serde.ColonSerde
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>     at java.lang.Class.forName0(Native Method)
>     at java.lang.Class.forName(Class.java:190)
>     at org.apache.hadoop.hive.ql.plan.TableDesc.getDeserializerClass(
> TableDesc.java:66)
>     ... 20 more
>
>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: Class not found exception from serde

Posted by Jason Dere <jd...@hortonworks.com>.
I'm assuming the jar was loaded using ADD JAR as opposed to the jar being on the classpath?

On Feb 24, 2014, at 2:40 PM, Jason Dere <jd...@hortonworks.com> wrote:

> I think TableDesc may not be using the right class loader here when it's trying to get the class - other parts of the code specify the class loader to use but in this context it does not.  I've opened https://issues.apache.org/jira/browse/HIVE-6495.
> 
> On Feb 23, 2014, at 4:05 AM, Andreas Koch <an...@gmail.com> wrote:
> 
>> 
>> I am trying to upgrade from Hive 0.10 to 0.12, but I am stuck trying to debug an error I keep getting. I have a couple of SerDes that I have developed, but when I try to query tables with these SerDes I get a ClassNotFoundException for the SerDe class. I know that the jar files have been loaded OK, because I can create tables with these SerDes, but when I try to query the table Hive throws an exception. I know that the SerDe interface that I use is now deprecated and I have also experimented with extending the AbstractSerDe class in stead, but I get the same behaviour.
>> 
>> Any ideas?
>> 
>> Thanks,
>> 
>> Andreas
>> 
>> 
>> FAILED: RuntimeException org.apache.hadoop.hive.ql.metadata.HiveException: Failed with exception java.lang.ClassNotFoundException: my.serde.ColonSerdejava.lang.RuntimeException: java.lang.ClassNotFoundException: my.serde.ColonSerde
>>     at org.apache.hadoop.hive.ql.plan.TableDesc.getDeserializerClass(TableDesc.java:68)
>>     at org.apache.hadoop.hive.ql.exec.FetchOperator.getRowInspectorFromTable(FetchOperator.java:231)
>>     at org.apache.hadoop.hive.ql.exec.FetchOperator.getOutputObjectInspector(FetchOperator.java:608)
>>     at org.apache.hadoop.hive.ql.exec.FetchTask.initialize(FetchTask.java:80)
>>     at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:497)
>>     at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:352)
>>     at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:995)
>>     at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1038)
>>     at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
>>     at org.apache.hadoop.hive.ql.Driver.run(Driver.java:921)
>>     at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
>>     at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
>>     at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:422)
>>     at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:790)
>>     at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)
>>     at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:623)
>>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>     at java.lang.reflect.Method.invoke(Method.java:606)
>>     at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
>> Caused by: java.lang.ClassNotFoundException: my.serde.ColonSerde
>>     at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>>     at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>>     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>>     at java.lang.Class.forName0(Native Method)
>>     at java.lang.Class.forName(Class.java:190)
>>     at org.apache.hadoop.hive.ql.plan.TableDesc.getDeserializerClass(TableDesc.java:66)
>>     ... 20 more
>> 
> 


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Class not found exception from serde

Posted by Jason Dere <jd...@hortonworks.com>.
I think TableDesc may not be using the right class loader here when it's trying to get the class - other parts of the code specify the class loader to use but in this context it does not.  I've opened https://issues.apache.org/jira/browse/HIVE-6495.

On Feb 23, 2014, at 4:05 AM, Andreas Koch <an...@gmail.com> wrote:

> 
> I am trying to upgrade from Hive 0.10 to 0.12, but I am stuck trying to debug an error I keep getting. I have a couple of SerDes that I have developed, but when I try to query tables with these SerDes I get a ClassNotFoundException for the SerDe class. I know that the jar files have been loaded OK, because I can create tables with these SerDes, but when I try to query the table Hive throws an exception. I know that the SerDe interface that I use is now deprecated and I have also experimented with extending the AbstractSerDe class in stead, but I get the same behaviour.
> 
> Any ideas?
> 
> Thanks,
> 
> Andreas
> 
> 
> FAILED: RuntimeException org.apache.hadoop.hive.ql.metadata.HiveException: Failed with exception java.lang.ClassNotFoundException: my.serde.ColonSerdejava.lang.RuntimeException: java.lang.ClassNotFoundException: my.serde.ColonSerde
>     at org.apache.hadoop.hive.ql.plan.TableDesc.getDeserializerClass(TableDesc.java:68)
>     at org.apache.hadoop.hive.ql.exec.FetchOperator.getRowInspectorFromTable(FetchOperator.java:231)
>     at org.apache.hadoop.hive.ql.exec.FetchOperator.getOutputObjectInspector(FetchOperator.java:608)
>     at org.apache.hadoop.hive.ql.exec.FetchTask.initialize(FetchTask.java:80)
>     at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:497)
>     at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:352)
>     at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:995)
>     at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1038)
>     at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
>     at org.apache.hadoop.hive.ql.Driver.run(Driver.java:921)
>     at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
>     at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
>     at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:422)
>     at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:790)
>     at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)
>     at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:623)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:606)
>     at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> Caused by: java.lang.ClassNotFoundException: my.serde.ColonSerde
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>     at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>     at java.lang.Class.forName0(Native Method)
>     at java.lang.Class.forName(Class.java:190)
>     at org.apache.hadoop.hive.ql.plan.TableDesc.getDeserializerClass(TableDesc.java:66)
>     ... 20 more
> 


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.