You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by Roland Bouman <ro...@gmail.com> on 2015/09/19 03:38:27 UTC
2nd phase of cube build always gets stuck with NPE in org.apache.kylin.job.tools.HadoopStatusGetter.get(HadoopStatusGetter.java:56)
Hi all,
my setup is a development machine with ubuntu 14.
I got:
- hadoop-2.7.1 (but also tried with hadoop-2.4.0)
- apache-hive-1.2.1-bin
- hbase-0.98.14-hadoop2
- kylin-1.0-incubating
I'm using a hive metastore that sits on a mysql database.
hadoop, hive and hbase all seem to be working like they should.
I setup the kylin sample, but when I build the cube it gets stuck at phase
2.
I get the same when I build a very simply cube myself with just one fact
table cube, no lookup tables and only normal dimensions, and only the count
measure.
This seems to be were it gets stuck:
[pool-4-thread-2]:[2015-09-19
03:15:26,021][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:92)]
- Job job_local404517651_0001 get status check result.
[pool-4-thread-2]:[2015-09-19
03:15:26,057][ERROR][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:91)]
- error check status
java.lang.NullPointerException
at
org.apache.kylin.job.tools.HadoopStatusGetter.get(HadoopStatusGetter.java:56)
at
org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:56)
at
org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:137)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
When I resume the job, I always get:
java.lang.NullPointerException
at
org.apache.kylin.job.common.MapReduceExecutable.onExecuteStart(MapReduceExecutable.java:74)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:106)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
I'm not sure whether these problems are related.
If anybody could recommend a solution I would be very grateful.
--
Roland Bouman
blog: http://rpbouman.blogspot.com/
twitter: @rolandbouman
linkedin: http://www.linkedin.com/profile/view?id=5142800&trk=tab_pro
Author of "Pentaho Solutions" (Wiley, ISBN: 978-0-470-48432-6
http://tinyurl.com/lvxa88) and "Pentaho Kettle Solutions" (Wiley, ISBN:
978-0-470-63517-9 http://tinyurl.com/33r7a8m)
Re: 2nd phase of cube build always gets stuck with NPE in org.apache.kylin.job.tools.HadoopStatusGetter.get(HadoopStatusGetter.java:56)
Posted by ShaoFeng Shi <sh...@gmail.com>.
Kylin wasn't able to get the job status from yarn; Not sure which caused
the NullPointerException, it might be a wrong URL, a more strict security
restriction, or a different message format; You can debug it or add more
logs in the HadoopStatusGetter；for exaple getting the status URL, and then
manually try that in web browser to see whether it returns the status
correctly;
2015-09-19 9:38 GMT+08:00 Roland Bouman <ro...@gmail.com>:
> Hi all,
>
> my setup is a development machine with ubuntu 14.
>
> I got:
> - hadoop-2.7.1 (but also tried with hadoop-2.4.0)
> - apache-hive-1.2.1-bin
> - hbase-0.98.14-hadoop2
> - kylin-1.0-incubating
>
> I'm using a hive metastore that sits on a mysql database.
>
> hadoop, hive and hbase all seem to be working like they should.
> I setup the kylin sample, but when I build the cube it gets stuck at phase
> 2.
> I get the same when I build a very simply cube myself with just one fact
> table cube, no lookup tables and only normal dimensions, and only the count
> measure.
>
> This seems to be were it gets stuck:
>
> [pool-4-thread-2]:[2015-09-19
>
> 03:15:26,021][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:92)]
> - Job job_local404517651_0001 get status check result.
>
> [pool-4-thread-2]:[2015-09-19
>
> 03:15:26,057][ERROR][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:91)]
> - error check status
> java.lang.NullPointerException
> at
>
> org.apache.kylin.job.tools.HadoopStatusGetter.get(HadoopStatusGetter.java:56)
> at
>
> org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:56)
> at
>
> org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:137)
> at
>
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
> at
>
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)
> at
>
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
> at
>
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
>
> When I resume the job, I always get:
>
> java.lang.NullPointerException
> at
>
> org.apache.kylin.job.common.MapReduceExecutable.onExecuteStart(MapReduceExecutable.java:74)
> at
>
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:106)
> at
>
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)
> at
>
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
> at
>
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
>
> I'm not sure whether these problems are related.
> If anybody could recommend a solution I would be very grateful.
>
> --
> Roland Bouman
> blog: http://rpbouman.blogspot.com/
> twitter: @rolandbouman
> linkedin: http://www.linkedin.com/profile/view?id=5142800&trk=tab_pro
>
> Author of "Pentaho Solutions" (Wiley, ISBN: 978-0-470-48432-6
> http://tinyurl.com/lvxa88) and "Pentaho Kettle Solutions" (Wiley, ISBN:
> 978-0-470-63517-9 http://tinyurl.com/33r7a8m)
>