You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kylin.apache.org by 蔡冉 <ca...@gmail.com> on 2016/06/12 09:08:34 UTC
Fwd: Can not load hive table in Kylin
I got error 'Oops... Failed to take action.' when trying to load hive table
in Kylin web ui, and no obvious error could be found in the log file,
anyone have met the problem before?
My environment:
Version:
hadoop: 2.7.1
hive: 1.2.1
hbase: 1.1.2
kylin: 1.5.2
We have total 20 physical servers in hadoop cluster, including 2 masters(
for HA) and 18 slaves.
Kylin installed in a docker container (for facilitating the test), and in
this docker container, whole of hadoop, hive and hbase installation dir
were copied from hadoop cluster servers. Commands like 'hadoop fs -ls',
'hive', 'hbase shell' can run and response successfully. Hive use a remote
mysql server for metastore, the configuration was properly specified in
hive-site.xml.
Attachment were the log files, any help would be appreciated.
Re: Can not load hive table in Kylin
Posted by ShaoFeng Shi <sh...@apache.org>.
Today in a customer's env, I encountered this problem; His environment
looks good: bin/find-hive-dependency.sh can identify the jars correctly;
but after Kylin start up, still got the ClassNotFound error:
java.lang.ClassNotFoundException:
org.apache.hadoop.hive.ql.session.SessionState
at
org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1720)
at
org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1571)
at
org.apache.kylin.rest.controller.TableController.showHiveDatabases(TableController.java:301)
Finally identified the root case is in hbase-env.sh; in which the customer
overwrited the "HBASE_CLASSPATH" variable, like:
export HBASE_CLASSPATH=/home/hadoop/hbase/conf
This is bad as it overwrites the variable totally, while Kylin depends on
it to tell JVM which jars need be on classpath. After changing it to:
export HBASE_CLASSPATH=/home/hadoop/hbase/conf:${HBASE_CLASSPATH}
Kylin can run well without the ClassNotFound error anymore.
2016-06-13 16:30 GMT+08:00 蔡冉 <ca...@gmail.com>:
> Thanks Shaofeng, I checked my chrome console and found below error, and I
> also found the error in tomcat log
> kylin/tomcat/logs/localhost.2016-06-13.log, which should be the root cause:
>
> org.springframework.web.util.NestedServletException: Handler processing
> failed; nested exception is java.lang.NoClassDefFoundError:
> org/apache/hadoop/hive/ql/session/SessionState
>
> but I can find hive-exec-1.2.1.jar in the result
> of find-hive-dependency.sh and the process, is there any other
> configuration I missed?
>
>
> 2016-06-12 22:05 GMT+08:00 ShaoFeng Shi <sh...@apache.org>:
>
>> It seems the request wasn't received by the server at all; Most likely be
>> a client side or network error; If you're using firefox or chrome, please
>> press F12 to open the console, and then check whether there is any JS or
>> HTTP error on the console.
>>
>> 2016-06-12 17:08 GMT+08:00 蔡冉 <ca...@gmail.com>:
>>
>>> I got error 'Oops... Failed to take action.' when trying to load hive
>>> table in Kylin web ui, and no obvious error could be found in the log file,
>>> anyone have met the problem before?
>>>
>>> My environment:
>>>
>>> Version:
>>> hadoop: 2.7.1
>>> hive: 1.2.1
>>> hbase: 1.1.2
>>> kylin: 1.5.2
>>>
>>> We have total 20 physical servers in hadoop cluster, including 2
>>> masters( for HA) and 18 slaves.
>>>
>>> Kylin installed in a docker container (for facilitating the test), and
>>> in this docker container, whole of hadoop, hive and hbase installation dir
>>> were copied from hadoop cluster servers. Commands like 'hadoop fs -ls',
>>> 'hive', 'hbase shell' can run and response successfully. Hive use a remote
>>> mysql server for metastore, the configuration was properly specified in
>>> hive-site.xml.
>>>
>>> Attachment were the log files, any help would be appreciated.
>>>
>>>
>>>
>>>
>>
>>
>> --
>> Best regards,
>>
>> Shaofeng Shi
>>
>>
>
--
Best regards,
Shaofeng Shi
Re: Can not load hive table in Kylin
Posted by Li Yang <li...@apache.org>.
Ran, you will want to debug the classpath that actually launches Kylin.
Find the below lines in kylin.sh and un-comment the second line. It should
print out what you need and you could start from there.
#debug if encounter NoClassDefError
#hbase classpath
Cheers
Yang
On Tue, Jun 14, 2016 at 5:39 PM, 蔡冉 <ca...@gmail.com> wrote:
> Hi Yang, I found my issue is similar with the one you resolved before,
> would you suggest the proper resolution? Thanks.
>
> The email thread and your reply before:
>
>
> https://mail-archives.apache.org/mod_mbox/kylin-dev/201508.mbox/%3CCAHRce1Nxq_R4xjc_VvFsnJWD99YjBJ1mFT7sj+mYF7p165u_bQ@mail.gmail.com%3E
>
> I visited 蜗牛. The problem is caused by HBASE_CLASSPATH being overwritten in hbase-config.sh. It's now fixed.
>
> For people who may encounter the same issue, `kylin.sh` exports HBASE_CLASSPATH and HBASE_CLASSPATH_PREFIX, then launch tomcat via `hbase` command. If the vars get overwritten by some config or env script inside `hbase`, you will get class not found issues like this.
>
>
> Regards,
> Ran Cai
>
> 2016-06-13 16:30 GMT+08:00 蔡冉 <ca...@gmail.com>:
>
>> Thanks Shaofeng, I checked my chrome console and found below error, and I
>> also found the error in tomcat log
>> kylin/tomcat/logs/localhost.2016-06-13.log, which should be the root cause:
>>
>> org.springframework.web.util.NestedServletException: Handler processing
>> failed; nested exception is java.lang.NoClassDefFoundError:
>> org/apache/hadoop/hive/ql/session/SessionState
>>
>> but I can find hive-exec-1.2.1.jar in the result
>> of find-hive-dependency.sh and the process, is there any other
>> configuration I missed?
>>
>>
>> 2016-06-12 22:05 GMT+08:00 ShaoFeng Shi <sh...@apache.org>:
>>
>>> It seems the request wasn't received by the server at all; Most likely
>>> be a client side or network error; If you're using firefox or chrome,
>>> please press F12 to open the console, and then check whether there is any
>>> JS or HTTP error on the console.
>>>
>>> 2016-06-12 17:08 GMT+08:00 蔡冉 <ca...@gmail.com>:
>>>
>>>> I got error 'Oops... Failed to take action.' when trying to load hive
>>>> table in Kylin web ui, and no obvious error could be found in the log file,
>>>> anyone have met the problem before?
>>>>
>>>> My environment:
>>>>
>>>> Version:
>>>> hadoop: 2.7.1
>>>> hive: 1.2.1
>>>> hbase: 1.1.2
>>>> kylin: 1.5.2
>>>>
>>>> We have total 20 physical servers in hadoop cluster, including 2
>>>> masters( for HA) and 18 slaves.
>>>>
>>>> Kylin installed in a docker container (for facilitating the test), and
>>>> in this docker container, whole of hadoop, hive and hbase installation dir
>>>> were copied from hadoop cluster servers. Commands like 'hadoop fs -ls',
>>>> 'hive', 'hbase shell' can run and response successfully. Hive use a remote
>>>> mysql server for metastore, the configuration was properly specified in
>>>> hive-site.xml.
>>>>
>>>> Attachment were the log files, any help would be appreciated.
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Best regards,
>>>
>>> Shaofeng Shi
>>>
>>>
>>
>
Re: Can not load hive table in Kylin
Posted by 蔡冉 <ca...@gmail.com>.
Hi Yang, I found my issue is similar with the one you resolved before,
would you suggest the proper resolution? Thanks.
The email thread and your reply before:
https://mail-archives.apache.org/mod_mbox/kylin-dev/201508.mbox/%3CCAHRce1Nxq_R4xjc_VvFsnJWD99YjBJ1mFT7sj+mYF7p165u_bQ@mail.gmail.com%3E
I visited 蜗牛. The problem is caused by HBASE_CLASSPATH being
overwritten in hbase-config.sh. It's now fixed.
For people who may encounter the same issue, `kylin.sh` exports
HBASE_CLASSPATH and HBASE_CLASSPATH_PREFIX, then launch tomcat via
`hbase` command. If the vars get overwritten by some config or env
script inside `hbase`, you will get class not found issues like this.
Regards,
Ran Cai
2016-06-13 16:30 GMT+08:00 蔡冉 <ca...@gmail.com>:
> Thanks Shaofeng, I checked my chrome console and found below error, and I
> also found the error in tomcat log
> kylin/tomcat/logs/localhost.2016-06-13.log, which should be the root cause:
>
> org.springframework.web.util.NestedServletException: Handler processing
> failed; nested exception is java.lang.NoClassDefFoundError:
> org/apache/hadoop/hive/ql/session/SessionState
>
> but I can find hive-exec-1.2.1.jar in the result
> of find-hive-dependency.sh and the process, is there any other
> configuration I missed?
>
>
> 2016-06-12 22:05 GMT+08:00 ShaoFeng Shi <sh...@apache.org>:
>
>> It seems the request wasn't received by the server at all; Most likely be
>> a client side or network error; If you're using firefox or chrome, please
>> press F12 to open the console, and then check whether there is any JS or
>> HTTP error on the console.
>>
>> 2016-06-12 17:08 GMT+08:00 蔡冉 <ca...@gmail.com>:
>>
>>> I got error 'Oops... Failed to take action.' when trying to load hive
>>> table in Kylin web ui, and no obvious error could be found in the log file,
>>> anyone have met the problem before?
>>>
>>> My environment:
>>>
>>> Version:
>>> hadoop: 2.7.1
>>> hive: 1.2.1
>>> hbase: 1.1.2
>>> kylin: 1.5.2
>>>
>>> We have total 20 physical servers in hadoop cluster, including 2
>>> masters( for HA) and 18 slaves.
>>>
>>> Kylin installed in a docker container (for facilitating the test), and
>>> in this docker container, whole of hadoop, hive and hbase installation dir
>>> were copied from hadoop cluster servers. Commands like 'hadoop fs -ls',
>>> 'hive', 'hbase shell' can run and response successfully. Hive use a remote
>>> mysql server for metastore, the configuration was properly specified in
>>> hive-site.xml.
>>>
>>> Attachment were the log files, any help would be appreciated.
>>>
>>>
>>>
>>>
>>
>>
>> --
>> Best regards,
>>
>> Shaofeng Shi
>>
>>
>
Re: Can not load hive table in Kylin
Posted by ShaoFeng Shi <sh...@apache.org>.
Today in a customer's env, I encountered this problem; His environment
looks good: bin/find-hive-dependency.sh can identify the jars correctly;
but after Kylin start up, still got the ClassNotFound error:
java.lang.ClassNotFoundException:
org.apache.hadoop.hive.ql.session.SessionState
at
org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1720)
at
org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1571)
at
org.apache.kylin.rest.controller.TableController.showHiveDatabases(TableController.java:301)
Finally identified the root case is in hbase-env.sh; in which the customer
overwrited the "HBASE_CLASSPATH" variable, like:
export HBASE_CLASSPATH=/home/hadoop/hbase/conf
This is bad as it overwrites the variable totally, while Kylin depends on
it to tell JVM which jars need be on classpath. After changing it to:
export HBASE_CLASSPATH=/home/hadoop/hbase/conf:${HBASE_CLASSPATH}
Kylin can run well without the ClassNotFound error anymore.
2016-06-13 16:30 GMT+08:00 蔡冉 <ca...@gmail.com>:
> Thanks Shaofeng, I checked my chrome console and found below error, and I
> also found the error in tomcat log
> kylin/tomcat/logs/localhost.2016-06-13.log, which should be the root cause:
>
> org.springframework.web.util.NestedServletException: Handler processing
> failed; nested exception is java.lang.NoClassDefFoundError:
> org/apache/hadoop/hive/ql/session/SessionState
>
> but I can find hive-exec-1.2.1.jar in the result
> of find-hive-dependency.sh and the process, is there any other
> configuration I missed?
>
>
> 2016-06-12 22:05 GMT+08:00 ShaoFeng Shi <sh...@apache.org>:
>
>> It seems the request wasn't received by the server at all; Most likely be
>> a client side or network error; If you're using firefox or chrome, please
>> press F12 to open the console, and then check whether there is any JS or
>> HTTP error on the console.
>>
>> 2016-06-12 17:08 GMT+08:00 蔡冉 <ca...@gmail.com>:
>>
>>> I got error 'Oops... Failed to take action.' when trying to load hive
>>> table in Kylin web ui, and no obvious error could be found in the log file,
>>> anyone have met the problem before?
>>>
>>> My environment:
>>>
>>> Version:
>>> hadoop: 2.7.1
>>> hive: 1.2.1
>>> hbase: 1.1.2
>>> kylin: 1.5.2
>>>
>>> We have total 20 physical servers in hadoop cluster, including 2
>>> masters( for HA) and 18 slaves.
>>>
>>> Kylin installed in a docker container (for facilitating the test), and
>>> in this docker container, whole of hadoop, hive and hbase installation dir
>>> were copied from hadoop cluster servers. Commands like 'hadoop fs -ls',
>>> 'hive', 'hbase shell' can run and response successfully. Hive use a remote
>>> mysql server for metastore, the configuration was properly specified in
>>> hive-site.xml.
>>>
>>> Attachment were the log files, any help would be appreciated.
>>>
>>>
>>>
>>>
>>
>>
>> --
>> Best regards,
>>
>> Shaofeng Shi
>>
>>
>
--
Best regards,
Shaofeng Shi
Re: Can not load hive table in Kylin
Posted by 蔡冉 <ca...@gmail.com>.
Thanks Shaofeng, I checked my chrome console and found below error, and I
also found the error in tomcat log
kylin/tomcat/logs/localhost.2016-06-13.log, which should be the root cause:
org.springframework.web.util.NestedServletException: Handler processing
failed; nested exception is java.lang.NoClassDefFoundError:
org/apache/hadoop/hive/ql/session/SessionState
but I can find hive-exec-1.2.1.jar in the result of find-hive-dependency.sh
and the process, is there any other configuration I missed?
2016-06-12 22:05 GMT+08:00 ShaoFeng Shi <sh...@apache.org>:
> It seems the request wasn't received by the server at all; Most likely be
> a client side or network error; If you're using firefox or chrome, please
> press F12 to open the console, and then check whether there is any JS or
> HTTP error on the console.
>
> 2016-06-12 17:08 GMT+08:00 蔡冉 <ca...@gmail.com>:
>
>> I got error 'Oops... Failed to take action.' when trying to load hive
>> table in Kylin web ui, and no obvious error could be found in the log file,
>> anyone have met the problem before?
>>
>> My environment:
>>
>> Version:
>> hadoop: 2.7.1
>> hive: 1.2.1
>> hbase: 1.1.2
>> kylin: 1.5.2
>>
>> We have total 20 physical servers in hadoop cluster, including 2 masters(
>> for HA) and 18 slaves.
>>
>> Kylin installed in a docker container (for facilitating the test), and in
>> this docker container, whole of hadoop, hive and hbase installation dir
>> were copied from hadoop cluster servers. Commands like 'hadoop fs -ls',
>> 'hive', 'hbase shell' can run and response successfully. Hive use a remote
>> mysql server for metastore, the configuration was properly specified in
>> hive-site.xml.
>>
>> Attachment were the log files, any help would be appreciated.
>>
>>
>>
>>
>
>
> --
> Best regards,
>
> Shaofeng Shi
>
>
Re: Can not load hive table in Kylin
Posted by ShaoFeng Shi <sh...@apache.org>.
It seems the request wasn't received by the server at all; Most likely be a
client side or network error; If you're using firefox or chrome, please
press F12 to open the console, and then check whether there is any JS or
HTTP error on the console.
2016-06-12 17:08 GMT+08:00 蔡冉 <ca...@gmail.com>:
> I got error 'Oops... Failed to take action.' when trying to load hive
> table in Kylin web ui, and no obvious error could be found in the log file,
> anyone have met the problem before?
>
> My environment:
>
> Version:
> hadoop: 2.7.1
> hive: 1.2.1
> hbase: 1.1.2
> kylin: 1.5.2
>
> We have total 20 physical servers in hadoop cluster, including 2 masters(
> for HA) and 18 slaves.
>
> Kylin installed in a docker container (for facilitating the test), and in
> this docker container, whole of hadoop, hive and hbase installation dir
> were copied from hadoop cluster servers. Commands like 'hadoop fs -ls',
> 'hive', 'hbase shell' can run and response successfully. Hive use a remote
> mysql server for metastore, the configuration was properly specified in
> hive-site.xml.
>
> Attachment were the log files, any help would be appreciated.
>
>
>
>
--
Best regards,
Shaofeng Shi