You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by 仇同心 <qi...@jd.com> on 2016/09/19 07:30:50 UTC
kylin-1.5.4同步hive元数据报错
大家好：
今天在使用kylin1.5.4版本时，在同步hive元数据时报错：
Load Hive Table Metadata From Tree页面一直显示：Loading Databases.
错误信息打印在kylin.out文件
SEVERE: Servlet.service() for servlet [kylin] in context with path [/kylin] threw exception [Handler processing failed;
nested exception is java.lang.NoClassDefFoundError: org/apache/hadoop/hive/ql/session/SessionState] with root cause
java.lang.ClassNotFoundException: org.apache.hadoop.hive.ql.session.SessionState
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1858)
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1701)
at org.apache.kylin.rest.controller.TableController.showHiveDatabases(TableController.java:315)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:13
6)
但是hive能正常使用，如果换成kylin 1.5.3版本没问题，Load Hive Table Metadata From Tree页面能显示出hive里的db.
我使用的hive版本是1.2.1，不知道kylin 1.5.4是否对hive 版本有要求？
谢谢！
Re: kylin-1.5.4同步hive元数据报错
Posted by hongbin ma <ma...@apache.org>.
cannot reproduce either
when you run kylin.sh, there will be stdout telling the extracted
hive/hbase dependency paths, can you attach them for further analysis?
On Tue, Sep 20, 2016 at 10:51 AM, Li Yang <li...@apache.org> wrote:
> We may not be able to reproduce the problem (at least I cannot). Both 1.5.4
> and 1.5.3 works for me.
>
> By my experience, the root cause is often HBASE_CLASSPATH was swallowed. As
> a test, try below in command shell.
>
> [root@sandbox]# export HBASE_CLASSPATH=*ABCDE*
> [root@sandbox]# hbase classpath
> /usr/hdp/2.2.4.2-2/hbase/conf:/usr/lib/jvm/java-1.7.0-
> openjdk.x86_64/lib/tools.jar:/usr/hdp/2.2.4.2-2/hbase:/usr/
> hdp/2.2.4.2-2/hbase/lib/activation-1.1.jar:/usr/hdp/2.
> 2.4.2-2/hbase/lib/aopalliance-1.0.jar.....:/usr/hdp/2.2.4.2-
> 2/zookeeper/*:/usr/hdp/2.2.4.2-2/zookeeper/lib/*:
> *ABCDE*
>
> If you don't get ABCDE from 'hbase classpath', that confirms
> HBASE_CLASSPATH was lost inside hbase shell.
>
> Cheers
> Yang
>
> On Tue, Sep 20, 2016 at 9:07 AM, ShaoFeng Shi <sh...@apache.org>
> wrote:
>
> > Hi Tongxin,
> >
> > 1.5.4 has no special requirement on hive version; From 1.5.3 to 1.5.4,
> the
> > kylin.sh has some change, please check whether it was the shell script
> > which wasn't able to detect the dependency jars correctly. Please share
> > with us about your finding, or if you can fix that and contribute a
> patch,
> > that would be great.
> >
> >
> >
> > 2016-09-19 15:30 GMT+08:00 仇同心 <qi...@jd.com>:
> >
> >> 大家好：
> >> 今天在使用kylin1.5.4版本时，在同步hive元数据时报错：
> >> Load Hive Table Metadata From Tree页面一直显示：Loading Databases.
> >>
> >>
> >> 错误信息打印在kylin.out文件
> >>
> >> SEVERE: Servlet.service() for servlet [kylin] in context with path
> >> [/kylin] threw exception [Handler processing failed;
> >> nested exception is java.lang.NoClassDefFoundError:
> >> org/apache/hadoop/hive/ql/session/SessionState] with root cause
> >> java.lang.ClassNotFoundException: org.apache.hadoop.hive.ql.sess
> >> ion.SessionState
> >> at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(W
> >> ebappClassLoaderBase.java:1858)
> >> at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(W
> >> ebappClassLoaderBase.java:1701)
> >> at org.apache.kylin.rest.controller.TableController.showHiveDat
> >> abases(TableController.java:315)
> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
> >> ssorImpl.java:57)
> >> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
> >> thodAccessorImpl.java:43)
> >> at java.lang.reflect.Method.invoke(Method.java:606)
> >> at org.springframework.web.method.support.InvocableHandlerMetho
> >> d.doInvoke(InvocableHandlerMethod.java:221)
> >> at org.springframework.web.method.support.InvocableHandlerMetho
> >> d.invokeForRequest(InvocableHandlerMethod.java:13
> >> 6)
> >>
> >> 但是hive能正常使用，如果换成kylin 1.5.3版本没问题，Load Hive Table Metadata From
> >> Tree页面能显示出hive里的db.
> >> 我使用的hive版本是1.2.1，不知道kylin 1.5.4是否对hive 版本有要求？
> >>
> >>
> >> 谢谢！
> >>
> >>
> >
> >
> > --
> > Best regards,
> >
> > Shaofeng Shi 史少锋
> >
> >
>
--
Regards,
*Bin Mahone | 马洪宾*
Re: kylin-1.5.4同步hive元数据报错
Posted by hongbin ma <ma...@apache.org>.
cannot reproduce either
when you run kylin.sh, there will be stdout telling the extracted
hive/hbase dependency paths, can you attach them for further analysis?
On Tue, Sep 20, 2016 at 10:51 AM, Li Yang <li...@apache.org> wrote:
> We may not be able to reproduce the problem (at least I cannot). Both 1.5.4
> and 1.5.3 works for me.
>
> By my experience, the root cause is often HBASE_CLASSPATH was swallowed. As
> a test, try below in command shell.
>
> [root@sandbox]# export HBASE_CLASSPATH=*ABCDE*
> [root@sandbox]# hbase classpath
> /usr/hdp/2.2.4.2-2/hbase/conf:/usr/lib/jvm/java-1.7.0-
> openjdk.x86_64/lib/tools.jar:/usr/hdp/2.2.4.2-2/hbase:/usr/
> hdp/2.2.4.2-2/hbase/lib/activation-1.1.jar:/usr/hdp/2.
> 2.4.2-2/hbase/lib/aopalliance-1.0.jar.....:/usr/hdp/2.2.4.2-
> 2/zookeeper/*:/usr/hdp/2.2.4.2-2/zookeeper/lib/*:
> *ABCDE*
>
> If you don't get ABCDE from 'hbase classpath', that confirms
> HBASE_CLASSPATH was lost inside hbase shell.
>
> Cheers
> Yang
>
> On Tue, Sep 20, 2016 at 9:07 AM, ShaoFeng Shi <sh...@apache.org>
> wrote:
>
> > Hi Tongxin,
> >
> > 1.5.4 has no special requirement on hive version; From 1.5.3 to 1.5.4,
> the
> > kylin.sh has some change, please check whether it was the shell script
> > which wasn't able to detect the dependency jars correctly. Please share
> > with us about your finding, or if you can fix that and contribute a
> patch,
> > that would be great.
> >
> >
> >
> > 2016-09-19 15:30 GMT+08:00 仇同心 <qi...@jd.com>:
> >
> >> 大家好：
> >> 今天在使用kylin1.5.4版本时，在同步hive元数据时报错：
> >> Load Hive Table Metadata From Tree页面一直显示：Loading Databases.
> >>
> >>
> >> 错误信息打印在kylin.out文件
> >>
> >> SEVERE: Servlet.service() for servlet [kylin] in context with path
> >> [/kylin] threw exception [Handler processing failed;
> >> nested exception is java.lang.NoClassDefFoundError:
> >> org/apache/hadoop/hive/ql/session/SessionState] with root cause
> >> java.lang.ClassNotFoundException: org.apache.hadoop.hive.ql.sess
> >> ion.SessionState
> >> at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(W
> >> ebappClassLoaderBase.java:1858)
> >> at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(W
> >> ebappClassLoaderBase.java:1701)
> >> at org.apache.kylin.rest.controller.TableController.showHiveDat
> >> abases(TableController.java:315)
> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
> >> ssorImpl.java:57)
> >> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
> >> thodAccessorImpl.java:43)
> >> at java.lang.reflect.Method.invoke(Method.java:606)
> >> at org.springframework.web.method.support.InvocableHandlerMetho
> >> d.doInvoke(InvocableHandlerMethod.java:221)
> >> at org.springframework.web.method.support.InvocableHandlerMetho
> >> d.invokeForRequest(InvocableHandlerMethod.java:13
> >> 6)
> >>
> >> 但是hive能正常使用，如果换成kylin 1.5.3版本没问题，Load Hive Table Metadata From
> >> Tree页面能显示出hive里的db.
> >> 我使用的hive版本是1.2.1，不知道kylin 1.5.4是否对hive 版本有要求？
> >>
> >>
> >> 谢谢！
> >>
> >>
> >
> >
> > --
> > Best regards,
> >
> > Shaofeng Shi 史少锋
> >
> >
>
--
Regards,
*Bin Mahone | 马洪宾*
Re: kylin-1.5.4同步hive元数据报错
Posted by Li Yang <li...@apache.org>.
We may not be able to reproduce the problem (at least I cannot). Both 1.5.4
and 1.5.3 works for me.
By my experience, the root cause is often HBASE_CLASSPATH was swallowed. As
a test, try below in command shell.
[root@sandbox]# export HBASE_CLASSPATH=*ABCDE*
[root@sandbox]# hbase classpath
/usr/hdp/2.2.4.2-2/hbase/conf:/usr/lib/jvm/java-1.7.0-openjdk.x86_64/lib/tools.jar:/usr/hdp/2.2.4.2-2/hbase:/usr/hdp/2.2.4.2-2/hbase/lib/activation-1.1.jar:/usr/hdp/2.2.4.2-2/hbase/lib/aopalliance-1.0.jar.....:/usr/hdp/2.2.4.2-2/zookeeper/*:/usr/hdp/2.2.4.2-2/zookeeper/lib/*:
*ABCDE*
If you don't get ABCDE from 'hbase classpath', that confirms
HBASE_CLASSPATH was lost inside hbase shell.
Cheers
Yang
On Tue, Sep 20, 2016 at 9:07 AM, ShaoFeng Shi <sh...@apache.org>
wrote:
> Hi Tongxin,
>
> 1.5.4 has no special requirement on hive version; From 1.5.3 to 1.5.4, the
> kylin.sh has some change, please check whether it was the shell script
> which wasn't able to detect the dependency jars correctly. Please share
> with us about your finding, or if you can fix that and contribute a patch,
> that would be great.
>
>
>
> 2016-09-19 15:30 GMT+08:00 仇同心 <qi...@jd.com>:
>
>> 大家好：
>> 今天在使用kylin1.5.4版本时，在同步hive元数据时报错：
>> Load Hive Table Metadata From Tree页面一直显示：Loading Databases.
>>
>>
>> 错误信息打印在kylin.out文件
>>
>> SEVERE: Servlet.service() for servlet [kylin] in context with path
>> [/kylin] threw exception [Handler processing failed;
>> nested exception is java.lang.NoClassDefFoundError:
>> org/apache/hadoop/hive/ql/session/SessionState] with root cause
>> java.lang.ClassNotFoundException: org.apache.hadoop.hive.ql.sess
>> ion.SessionState
>> at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(W
>> ebappClassLoaderBase.java:1858)
>> at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(W
>> ebappClassLoaderBase.java:1701)
>> at org.apache.kylin.rest.controller.TableController.showHiveDat
>> abases(TableController.java:315)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>> ssorImpl.java:57)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>> thodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at org.springframework.web.method.support.InvocableHandlerMetho
>> d.doInvoke(InvocableHandlerMethod.java:221)
>> at org.springframework.web.method.support.InvocableHandlerMetho
>> d.invokeForRequest(InvocableHandlerMethod.java:13
>> 6)
>>
>> 但是hive能正常使用，如果换成kylin 1.5.3版本没问题，Load Hive Table Metadata From
>> Tree页面能显示出hive里的db.
>> 我使用的hive版本是1.2.1，不知道kylin 1.5.4是否对hive 版本有要求？
>>
>>
>> 谢谢！
>>
>>
>
>
> --
> Best regards,
>
> Shaofeng Shi 史少锋
>
>
Re: kylin-1.5.4同步hive元数据报错
Posted by Li Yang <li...@apache.org>.
We may not be able to reproduce the problem (at least I cannot). Both 1.5.4
and 1.5.3 works for me.
By my experience, the root cause is often HBASE_CLASSPATH was swallowed. As
a test, try below in command shell.
[root@sandbox]# export HBASE_CLASSPATH=*ABCDE*
[root@sandbox]# hbase classpath
/usr/hdp/2.2.4.2-2/hbase/conf:/usr/lib/jvm/java-1.7.0-openjdk.x86_64/lib/tools.jar:/usr/hdp/2.2.4.2-2/hbase:/usr/hdp/2.2.4.2-2/hbase/lib/activation-1.1.jar:/usr/hdp/2.2.4.2-2/hbase/lib/aopalliance-1.0.jar.....:/usr/hdp/2.2.4.2-2/zookeeper/*:/usr/hdp/2.2.4.2-2/zookeeper/lib/*:
*ABCDE*
If you don't get ABCDE from 'hbase classpath', that confirms
HBASE_CLASSPATH was lost inside hbase shell.
Cheers
Yang
On Tue, Sep 20, 2016 at 9:07 AM, ShaoFeng Shi <sh...@apache.org>
wrote:
> Hi Tongxin,
>
> 1.5.4 has no special requirement on hive version; From 1.5.3 to 1.5.4, the
> kylin.sh has some change, please check whether it was the shell script
> which wasn't able to detect the dependency jars correctly. Please share
> with us about your finding, or if you can fix that and contribute a patch,
> that would be great.
>
>
>
> 2016-09-19 15:30 GMT+08:00 仇同心 <qi...@jd.com>:
>
>> 大家好：
>> 今天在使用kylin1.5.4版本时，在同步hive元数据时报错：
>> Load Hive Table Metadata From Tree页面一直显示：Loading Databases.
>>
>>
>> 错误信息打印在kylin.out文件
>>
>> SEVERE: Servlet.service() for servlet [kylin] in context with path
>> [/kylin] threw exception [Handler processing failed;
>> nested exception is java.lang.NoClassDefFoundError:
>> org/apache/hadoop/hive/ql/session/SessionState] with root cause
>> java.lang.ClassNotFoundException: org.apache.hadoop.hive.ql.sess
>> ion.SessionState
>> at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(W
>> ebappClassLoaderBase.java:1858)
>> at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(W
>> ebappClassLoaderBase.java:1701)
>> at org.apache.kylin.rest.controller.TableController.showHiveDat
>> abases(TableController.java:315)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>> ssorImpl.java:57)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>> thodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at org.springframework.web.method.support.InvocableHandlerMetho
>> d.doInvoke(InvocableHandlerMethod.java:221)
>> at org.springframework.web.method.support.InvocableHandlerMetho
>> d.invokeForRequest(InvocableHandlerMethod.java:13
>> 6)
>>
>> 但是hive能正常使用，如果换成kylin 1.5.3版本没问题，Load Hive Table Metadata From
>> Tree页面能显示出hive里的db.
>> 我使用的hive版本是1.2.1，不知道kylin 1.5.4是否对hive 版本有要求？
>>
>>
>> 谢谢！
>>
>>
>
>
> --
> Best regards,
>
> Shaofeng Shi 史少锋
>
>
Re: kylin-1.5.4同步hive元数据报错
Posted by ShaoFeng Shi <sh...@apache.org>.
Hi Tongxin,
1.5.4 has no special requirement on hive version; From 1.5.3 to 1.5.4, the
kylin.sh has some change, please check whether it was the shell script
which wasn't able to detect the dependency jars correctly. Please share
with us about your finding, or if you can fix that and contribute a patch,
that would be great.
2016-09-19 15:30 GMT+08:00 仇同心 <qi...@jd.com>:
> 大家好：
> 今天在使用kylin1.5.4版本时，在同步hive元数据时报错：
> Load Hive Table Metadata From Tree页面一直显示：Loading Databases.
>
>
> 错误信息打印在kylin.out文件
>
> SEVERE: Servlet.service() for servlet [kylin] in context with path
> [/kylin] threw exception [Handler processing failed;
> nested exception is java.lang.NoClassDefFoundError:
> org/apache/hadoop/hive/ql/session/SessionState] with root cause
> java.lang.ClassNotFoundException: org.apache.hadoop.hive.ql.
> session.SessionState
> at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(
> WebappClassLoaderBase.java:1858)
> at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(
> WebappClassLoaderBase.java:1701)
> at org.apache.kylin.rest.controller.TableController.
> showHiveDatabases(TableController.java:315)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.springframework.web.method.support.InvocableHandlerMethod.
> doInvoke(InvocableHandlerMethod.java:221)
> at org.springframework.web.method.support.InvocableHandlerMethod.
> invokeForRequest(InvocableHandlerMethod.java:13
> 6)
>
> 但是hive能正常使用，如果换成kylin 1.5.3版本没问题，Load Hive Table Metadata From
> Tree页面能显示出hive里的db.
> 我使用的hive版本是1.2.1，不知道kylin 1.5.4是否对hive 版本有要求？
>
>
> 谢谢！
>
>
--
Best regards,
Shaofeng Shi 史少锋
Re: kylin-1.5.4同步hive元数据报错
Posted by ShaoFeng Shi <sh...@apache.org>.
Hi Tongxin,
1.5.4 has no special requirement on hive version; From 1.5.3 to 1.5.4, the
kylin.sh has some change, please check whether it was the shell script
which wasn't able to detect the dependency jars correctly. Please share
with us about your finding, or if you can fix that and contribute a patch,
that would be great.
2016-09-19 15:30 GMT+08:00 仇同心 <qi...@jd.com>:
> 大家好：
> 今天在使用kylin1.5.4版本时，在同步hive元数据时报错：
> Load Hive Table Metadata From Tree页面一直显示：Loading Databases.
>
>
> 错误信息打印在kylin.out文件
>
> SEVERE: Servlet.service() for servlet [kylin] in context with path
> [/kylin] threw exception [Handler processing failed;
> nested exception is java.lang.NoClassDefFoundError:
> org/apache/hadoop/hive/ql/session/SessionState] with root cause
> java.lang.ClassNotFoundException: org.apache.hadoop.hive.ql.
> session.SessionState
> at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(
> WebappClassLoaderBase.java:1858)
> at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(
> WebappClassLoaderBase.java:1701)
> at org.apache.kylin.rest.controller.TableController.
> showHiveDatabases(TableController.java:315)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.springframework.web.method.support.InvocableHandlerMethod.
> doInvoke(InvocableHandlerMethod.java:221)
> at org.springframework.web.method.support.InvocableHandlerMethod.
> invokeForRequest(InvocableHandlerMethod.java:13
> 6)
>
> 但是hive能正常使用，如果换成kylin 1.5.3版本没问题，Load Hive Table Metadata From
> Tree页面能显示出hive里的db.
> 我使用的hive版本是1.2.1，不知道kylin 1.5.4是否对hive 版本有要求？
>
>
> 谢谢！
>
>
--
Best regards,
Shaofeng Shi 史少锋