You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kylin.apache.org by Mehak Mehta <me...@stonybrook.edu> on 2015/12/08 04:46:54 UTC
Nullpointer Exception in Kylin while checking the status of cube
Hi,
I am getting the following error while building my cube using browser
console on my Hivetable.
I am using default architecture
hadoop-2.6.0 (cluster of 10 nodes)
apache-hive-0.14.0 (using default metastore)
apache-kylin-1.1.1 (on single node)
hbase-0.98.16 (on single node as pseudo cluster)
Note I have recompiled hadoop (with Snappy library) for my system (Centos
6.5 64 bit).
It starts building the cube but fails while checking its status with the
following *Kylin logs,* can you please help me:
015-12-05 03:34:35,779 INFO [Thread-36] mapred.LocalJobRunner:
OutputCommitter is
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
2015-12-05 03:34:35,848 INFO [Thread-36] mapred.LocalJobRunner: Waiting
for map tasks
2015-12-05 03:34:35,849 INFO [LocalJobRunner Map Task Executor #0]
mapred.LocalJobRunner: Starting task:
attempt_local1931418873_0001_m_000000_0
2015-12-05 03:34:35,890 INFO [LocalJobRunner Map Task Executor #0]
mapred.Task: Using ResourceCalculatorProcessTree : [ ]
2015-12-05 03:34:35,897 INFO [LocalJobRunner Map Task Executor #0]
mapred.MapTask: Processing split:
org.apache.hive.hcatalog.mapreduce.HCatSplit@7d6447d
2015-12-05 03:34:35,921 INFO [LocalJobRunner Map Task Executor #0]
mapred.MapTask: Map output collector class =
org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
mapred.MapTask: mapreduce.task.io.sort.mb: 100
2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
mapred.MapTask: soft limit at 83886080
2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
mapred.MapTask: bufstart = 0; bufvoid = 104857600
2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
mapred.MapTask: kvstart = 26214396; length = 6553600
[pool-4-thread-2]:[2015-12-05
03:34:35,966][WARN][org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:682)]
- Going to buffer response body of large or unknown size. Using
getResponseBodyAsStream instead is recommended.
[pool-4-thread-2]:[2015-12-05
03:34:35,967][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:92)]
- Job job_local1931418873_0001 get status check result.
[pool-4-thread-2]:[2015-12-05
03:34:35,994][ERROR][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:91)]
- error check status
*java.lang.NullPointerException*
* at
org.apache.kylin.job.tools.HadoopStatusGetter.get(HadoopStatusGetter.java:56)*
* at
org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:56)*
* at
org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:137)*
* at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
* at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)*
* at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
* at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)*
* at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)*
* at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)*
* at java.lang.Thread.run(Thread.java:745)*
[pool-4-thread-2]:[2015-12-05
03:34:35,998][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
- Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
(Store kylin_metadata@hbase)
[pool-4-thread-2]:[2015-12-05
03:34:36,003][DEBUG][org.apache.kylin.job.common.HadoopCmdOutput.updateJobCounter(HadoopCmdOutput.java:94)]
- Counters: 0
2015-12-05 03:34:36,010 WARN [pool-4-thread-2] mapreduce.Counters: Group
FileSystemCounters is deprecated. Use
org.apache.hadoop.mapreduce.FileSystemCounter instead
[pool-4-thread-2]:[2015-12-05
03:34:36,013][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
- Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
(Store kylin_metadata@hbase)
[pool-4-thread-2]:[2015-12-05
03:34:36,018][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
- Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
(Store kylin_metadata@hbase)
[pool-4-thread-2]:[2015-12-05
03:34:36,023][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
- Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
(Store kylin_metadata@hbase)
I couldn't see any error in my Hive logs or Hadoop logs, but there were
some logs related to this in *Hbase Logs:*
2015-12-05 03:29:43,208 DEBUG [region-location-2]
regionserver.HRegionFileSystem: No StoreFiles for:
file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/a
*2015-12-05 03:29:43,208 DEBUG [region-location-2]
regionserver.HRegionFileSystem: No StoreFiles for:
file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/q*
*2015-12-05 03:30:01,382 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181
<http://0.0.0.0/0.0.0.0:2181>] server.NIOServerCnxnFactory: Accepted socket
connection from /127.0.0.1:36859 <http://127.0.0.1:36859/>*
*2015-12-05 03:30:01,387 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181
<http://0.0.0.0/0.0.0.0:2181>] server.ZooKeeperServer: Client attempting to
establish new session at /127.0.0.1:36859 <http://127.0.0.1:36859/>*
*2015-12-05 03:30:01,411 INFO [SyncThread:0] server.ZooKeeperServer:
Established session 0x15171398d090008 with negotiated timeout 40000 for
client /127.0.0.1:36859 <http://127.0.0.1:36859/>*
*2015-12-05 03:30:03,761 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181
<http://0.0.0.0/0.0.0.0:2181>] server.NIOServerCnxnFactory: Accepted socket
connection from /127.0.0.1:36862 <http://127.0.0.1:36862/>*
*2015-12-05 03:30:03,761 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181
<http://0.0.0.0/0.0.0.0:2181>] server.ZooKeeperServer: Client attempting to
establish new session at /127.0.0.1:36862 <http://127.0.0.1:36862/>*
*2015-12-05 03:30:03,763 INFO [SyncThread:0] server.ZooKeeperServer:
Established session 0x15171398d090009 with negotiated timeout 40000 for
client /127.0.0.1:36862 <http://127.0.0.1:36862/>*
*2015-12-05 03:30:03,816 INFO [ProcessThread(sid:0 cport:-1):]
server.PrepRequestProcessor: Got user-level KeeperException when processing
sessionid:0x15171398d090009 type:create cxid:0x1 zxid:0x52d txntype:-1
reqpath:n/a Error Path:/kylin/job_engine/lock/kylin_metadata
Error:KeeperErrorCode = NoNode for /kylin/job_engine/lock/kylin_metadata*
*2015-12-05 03:30:43,210 DEBUG [region-location-3]
regionserver.HRegionFileSystem: No StoreFiles for:
file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/a*
*2015-12-05 03:30:43,211 DEBUG [region-location-3]
regionserver.HRegionFileSystem: No StoreFiles for:
file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/q*
*2015-12-05 03:31:43,210 DEBUG [region-location-4]
regionserver.HRegionFileSystem: No StoreFiles for:
file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/a*
*2015-12-05 03:31:43,210 DEBUG [region-location-4]
regionserver.HRegionFileSystem: No StoreFiles for:
file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/q*
But I couldn't figure out the cause of the problem, can anybody please help
me in resolving this.
I have been struggling with this for sometime now.
--
Thanks & Regards
Mehak Mehta
Masters - Computer Science
Re: Nullpointer Exception in Kylin while checking the status of cube
Posted by Li Yang <li...@apache.org>.
It's likely that Kylin did extract from hive but pulled 0 records out of
it. Click the 'parameters' icon (like a key) in the first job step "Create
Intermediate Flat Hive Table" and find the exact command that Kylin used to
extract from hive. Like
.....
CREATE EXTERNAL TABLE IF NOT EXISTS
kylin_intermediate_xxxxx_20150101000000_20150927000000
.....
INSERT OVERWRITE TABLE
kylin_intermediate_xxxxx_20150101000000_20150927000000 SELECT ....
.....
Double check the command does return some records.
On Thu, Dec 17, 2015 at 2:43 PM, Mehak Mehta <me...@stonybrook.edu>
wrote:
> Hi Li,
>
> Again thanks for your reply. I resolved that issue and I am able to build
> the cube.
> But after completion of building it. After building it my Cube size and
> Source file is shown as 0.
> As shown below.
>
> [image: Inline image 1]
>
> How can I check/debug this problem.
> Can you help me with this.
>
> I have checked logs and Kylin is retrieving data from Hive tables.
>
> Thanks & Regards,
> Mehak
>
>
>
> On Mon, Dec 14, 2015 at 3:29 AM, Li Yang <li...@apache.org> wrote:
>
>> Pls check https://issues.apache.org/jira/browse/KYLIN-953
>>
>>
>> On Fri, Dec 11, 2015 at 8:25 PM, Mehak Mehta <me...@stonybrook.edu>
>> wrote:
>>
>>> Hi,
>>>
>>> Thanks Li Yang for your reply.
>>>
>>> I solved that previous issue.
>>> Now my cube is building but it always fails at 70% with the following
>>> error:
>>>
>>> 2015-12-11 07:12:37,021 INFO [pool-4-thread-1]
>>> mapreduce.HFileOutputFormat2: bulkload locality sensitive enabled
>>> 2015-12-11 07:12:37,021 INFO [pool-4-thread-1]
>>> mapreduce.HFileOutputFormat2: Looking up current regions for table
>>> KYLIN_23ZG714AUU
>>> 2015-12-11 07:12:37,029 INFO [pool-4-thread-1]
>>> mapreduce.HFileOutputFormat2: Configuring 1 reduce partitions to match
>>> current region count
>>> [pool-4-thread-1]:[2015-12-11 07:12:37,030][ERROR]*[org.apache.kylin.job.hadoop.cube.CubeHFileJob.run(CubeHFileJob.java:95)]
>>> - error in CubeHFileJob*
>>> *java.lang.IllegalArgumentException: Can not create a Path from a null
>>> string*
>>> * at org.apache.hadoop.fs.Path.checkPathArg(Path.java:123)*
>>> * at org.apache.hadoop.fs.Path.<init>(Path.java:135)*
>>> * at org.apache.hadoop.fs.Path.<init>(Path.java:89)*
>>> * at
>>> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configurePartitioner(HFileOutputFormat2.java:615)*
>>> * at
>>> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad(HFileOutputFormat2.java:464)*
>>> * at
>>> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.configureIncrementalLoad(HFileOutputFormat.java:88)*
>>> * at
>>> org.apache.kylin.job.hadoop.cube.CubeHFileJob.run(CubeHFileJob.java:86)*
>>> * at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)*
>>> * at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)*
>>> * at
>>> org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:113)*
>>> * at
>>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
>>> * at
>>> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)*
>>> * at
>>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
>>> * at
>>> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)*
>>> * at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)*
>>> * at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)*
>>> * at java.lang.Thread.run(Thread.java:745)*
>>> *usage: CubeHFileJob*
>>> * -cubename <name> Cube name. For exmaple, flat_item_cube*
>>> * -htablename <htable name> HTable name*
>>> * -input <path> Input path*
>>> * -jobname <name> Job name. For exmaple,*
>>> * Kylin_Cuboid_Builder-clsfd_v2_Step_22-D)*
>>> -output <path> Output path
>>> [pool-4-thread-1]:[2015-12-11
>>> 07:12:37,034][INFO][org.apache.kylin.job.hadoop.AbstractHadoopJob.cleanupTempConfFile(AbstractHadoopJob.java:340)]
>>> - tempMetaFileString is :
>>> file:///data/shared/mehak/hadoop/kylin/./bin/../tomcat/temp/kylin_job_meta4467707123004651766/meta
>>> *[pool-4-thread-1]:[2015-12-11
>>> 07:12:37,040][ERROR][org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:116)]
>>> - error execute
>>> MapReduceExecutable{id=dbf394d0-5a18-4299-b191-c7bc0dc9b941-09,
>>> name=Convert Cuboid Data to HFile, state=RUNNING}*
>>> *java.lang.IllegalArgumentException: Can not create a Path from a null
>>> string*
>>> * at org.apache.hadoop.fs.Path.checkPathArg(Path.java:123)*
>>> * at org.apache.hadoop.fs.Path.<init>(Path.java:135)*
>>> * at org.apache.hadoop.fs.Path.<init>(Path.java:89)*
>>> * at
>>> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configurePartitioner(HFileOutputFormat2.java:615)*
>>> * at
>>> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad(HFileOutputFormat2.java:464)*
>>> * at
>>> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.configureIncrementalLoad(HFileOutputFormat.java:88)*
>>> * at
>>> org.apache.kylin.job.hadoop.cube.CubeHFileJob.run(CubeHFileJob.java:86)*
>>> * at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)*
>>> * at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)*
>>> * at
>>> org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:113)*
>>> * at
>>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
>>> * at
>>> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)*
>>> * at
>>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
>>> * at
>>> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)*
>>> * at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)*
>>> * at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)*
>>> * at java.lang.Thread.run(Thread.java:745)*
>>> [pool-4-thread-1]:[2015-12-11
>>> 07:12:37,045][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>>> - Saving resource /execute_output/dbf394d0-5a18-4299-b191-c7bc0dc9b941-09
>>> (Store kylin_metadata@hbase)
>>> [pool-4-thread-1]:[2015-12-11
>>> 07:12:37,051][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>>> - Saving resource /execute_output/dbf394d0-5a18-4299-b191-c7bc0dc9b941-09
>>> (Store kylin_metadata@hbase)
>>> [pool-4-thread-1]:[2015-12-11
>>> 07:12:37,053][INFO][org.apache.kylin.job.manager.ExecutableManager.updateJobOutput(ExecutableManager.java:241)]
>>> - job id:dbf394d0-5a18-4299-b191-c7bc0dc9b941-09 from RUNNING to ERROR
>>>
>>> I cannot see any errors in hadoop, hive or hbase logs. It seems to be
>>> some configuration issue.
>>> Kindly help, I have tried a lot configurations but it doesn't work.
>>>
>>> Thanks & Regards,
>>> Mehak Mehta
>>>
>>>
>>>
>>> On Thu, Dec 10, 2015 at 4:42 AM, Li Yang <li...@apache.org> wrote:
>>>
>>>> Kylin depends on a hadoop config "yarn.resourcemanager.webapp.address"
>>>> to find resource manager and come up a ws URL to query job status.
>>>>
>>>> You can check if the config presents in site xmls on Kylin classpath.
>>>>
>>>> Or if troubleshooting Hadoop is too painful, a simpler way is adding a
>>>> line in conf/kylin.properties.
>>>>
>>>> kylin.job.yarn.app.rest.check.status.url=
>>>> https://YOUR_RM_AND_PORT/ws/v1/cluster/apps/${job_id}?anonymous=true
>>>>
>>>> On Tue, Dec 8, 2015 at 11:46 AM, Mehak Mehta <
>>>> mehak.mehta@stonybrook.edu> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I am getting the following error while building my cube using browser
>>>>> console on my Hivetable.
>>>>> I am using default architecture
>>>>>
>>>>> hadoop-2.6.0 (cluster of 10 nodes)
>>>>> apache-hive-0.14.0 (using default metastore)
>>>>> apache-kylin-1.1.1 (on single node)
>>>>> hbase-0.98.16 (on single node as pseudo cluster)
>>>>>
>>>>> Note I have recompiled hadoop (with Snappy library) for my system
>>>>> (Centos 6.5 64 bit).
>>>>> It starts building the cube but fails while checking its status with
>>>>> the following *Kylin logs,* can you please help me:
>>>>>
>>>>> 015-12-05 03:34:35,779 INFO [Thread-36] mapred.LocalJobRunner:
>>>>> OutputCommitter is
>>>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
>>>>> 2015-12-05 03:34:35,848 INFO [Thread-36] mapred.LocalJobRunner:
>>>>> Waiting for map tasks
>>>>> 2015-12-05 03:34:35,849 INFO [LocalJobRunner Map Task Executor #0]
>>>>> mapred.LocalJobRunner: Starting task:
>>>>> attempt_local1931418873_0001_m_000000_0
>>>>> 2015-12-05 03:34:35,890 INFO [LocalJobRunner Map Task Executor #0]
>>>>> mapred.Task: Using ResourceCalculatorProcessTree : [ ]
>>>>> 2015-12-05 03:34:35,897 INFO [LocalJobRunner Map Task Executor #0]
>>>>> mapred.MapTask: Processing split:
>>>>> org.apache.hive.hcatalog.mapreduce.HCatSplit@7d6447d
>>>>> 2015-12-05 03:34:35,921 INFO [LocalJobRunner Map Task Executor #0]
>>>>> mapred.MapTask: Map output collector class =
>>>>> org.apache.hadoop.mapred.MapTask$MapOutputBuffer
>>>>> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
>>>>> mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
>>>>> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
>>>>> mapred.MapTask: mapreduce.task.io.sort.mb: 100
>>>>> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
>>>>> mapred.MapTask: soft limit at 83886080
>>>>> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
>>>>> mapred.MapTask: bufstart = 0; bufvoid = 104857600
>>>>> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
>>>>> mapred.MapTask: kvstart = 26214396; length = 6553600
>>>>> [pool-4-thread-2]:[2015-12-05
>>>>> 03:34:35,966][WARN][org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:682)]
>>>>> - Going to buffer response body of large or unknown size. Using
>>>>> getResponseBodyAsStream instead is recommended.
>>>>> [pool-4-thread-2]:[2015-12-05
>>>>> 03:34:35,967][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:92)]
>>>>> - Job job_local1931418873_0001 get status check result.
>>>>>
>>>>> [pool-4-thread-2]:[2015-12-05
>>>>> 03:34:35,994][ERROR][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:91)]
>>>>> - error check status
>>>>> *java.lang.NullPointerException*
>>>>> * at
>>>>> org.apache.kylin.job.tools.HadoopStatusGetter.get(HadoopStatusGetter.java:56)*
>>>>> * at
>>>>> org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:56)*
>>>>> * at
>>>>> org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:137)*
>>>>> * at
>>>>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
>>>>> * at
>>>>> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)*
>>>>> * at
>>>>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
>>>>> * at
>>>>> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)*
>>>>> * at
>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)*
>>>>> * at
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)*
>>>>> * at java.lang.Thread.run(Thread.java:745)*
>>>>> [pool-4-thread-2]:[2015-12-05
>>>>> 03:34:35,998][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>>>>> - Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
>>>>> (Store kylin_metadata@hbase)
>>>>> [pool-4-thread-2]:[2015-12-05
>>>>> 03:34:36,003][DEBUG][org.apache.kylin.job.common.HadoopCmdOutput.updateJobCounter(HadoopCmdOutput.java:94)]
>>>>> - Counters: 0
>>>>> 2015-12-05 03:34:36,010 WARN [pool-4-thread-2] mapreduce.Counters:
>>>>> Group FileSystemCounters is deprecated. Use
>>>>> org.apache.hadoop.mapreduce.FileSystemCounter instead
>>>>> [pool-4-thread-2]:[2015-12-05
>>>>> 03:34:36,013][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>>>>> - Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
>>>>> (Store kylin_metadata@hbase)
>>>>> [pool-4-thread-2]:[2015-12-05
>>>>> 03:34:36,018][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>>>>> - Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
>>>>> (Store kylin_metadata@hbase)
>>>>> [pool-4-thread-2]:[2015-12-05
>>>>> 03:34:36,023][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>>>>> - Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
>>>>> (Store kylin_metadata@hbase)
>>>>>
>>>>>
>>>>>
>>>>> I couldn't see any error in my Hive logs or Hadoop logs, but there
>>>>> were some logs related to this in *Hbase Logs:*
>>>>>
>>>>> 2015-12-05 03:29:43,208 DEBUG [region-location-2]
>>>>> regionserver.HRegionFileSystem: No StoreFiles for:
>>>>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/a
>>>>> *2015-12-05 03:29:43,208 DEBUG [region-location-2]
>>>>> regionserver.HRegionFileSystem: No StoreFiles for:
>>>>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/q*
>>>>> *2015-12-05 03:30:01,382 INFO
>>>>> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181 <http://0.0.0.0/0.0.0.0:2181>]
>>>>> server.NIOServerCnxnFactory: Accepted socket connection from
>>>>> /127.0.0.1:36859 <http://127.0.0.1:36859/>*
>>>>> *2015-12-05 03:30:01,387 INFO
>>>>> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181 <http://0.0.0.0/0.0.0.0:2181>]
>>>>> server.ZooKeeperServer: Client attempting to establish new session at
>>>>> /127.0.0.1:36859 <http://127.0.0.1:36859/>*
>>>>> *2015-12-05 03:30:01,411 INFO [SyncThread:0] server.ZooKeeperServer:
>>>>> Established session 0x15171398d090008 with negotiated timeout 40000 for
>>>>> client /127.0.0.1:36859 <http://127.0.0.1:36859/>*
>>>>> *2015-12-05 03:30:03,761 INFO
>>>>> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181 <http://0.0.0.0/0.0.0.0:2181>]
>>>>> server.NIOServerCnxnFactory: Accepted socket connection from
>>>>> /127.0.0.1:36862 <http://127.0.0.1:36862/>*
>>>>> *2015-12-05 03:30:03,761 INFO
>>>>> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181 <http://0.0.0.0/0.0.0.0:2181>]
>>>>> server.ZooKeeperServer: Client attempting to establish new session at
>>>>> /127.0.0.1:36862 <http://127.0.0.1:36862/>*
>>>>> *2015-12-05 03:30:03,763 INFO [SyncThread:0] server.ZooKeeperServer:
>>>>> Established session 0x15171398d090009 with negotiated timeout 40000 for
>>>>> client /127.0.0.1:36862 <http://127.0.0.1:36862/>*
>>>>> *2015-12-05 03:30:03,816 INFO [ProcessThread(sid:0 cport:-1):]
>>>>> server.PrepRequestProcessor: Got user-level KeeperException when processing
>>>>> sessionid:0x15171398d090009 type:create cxid:0x1 zxid:0x52d txntype:-1
>>>>> reqpath:n/a Error Path:/kylin/job_engine/lock/kylin_metadata
>>>>> Error:KeeperErrorCode = NoNode for /kylin/job_engine/lock/kylin_metadata*
>>>>> *2015-12-05 03:30:43,210 DEBUG [region-location-3]
>>>>> regionserver.HRegionFileSystem: No StoreFiles for:
>>>>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/a*
>>>>> *2015-12-05 03:30:43,211 DEBUG [region-location-3]
>>>>> regionserver.HRegionFileSystem: No StoreFiles for:
>>>>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/q*
>>>>> *2015-12-05 03:31:43,210 DEBUG [region-location-4]
>>>>> regionserver.HRegionFileSystem: No StoreFiles for:
>>>>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/a*
>>>>> *2015-12-05 03:31:43,210 DEBUG [region-location-4]
>>>>> regionserver.HRegionFileSystem: No StoreFiles for:
>>>>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/q*
>>>>>
>>>>> But I couldn't figure out the cause of the problem, can anybody please
>>>>> help me in resolving this.
>>>>> I have been struggling with this for sometime now.
>>>>>
>>>>> --
>>>>> Thanks & Regards
>>>>> Mehak Mehta
>>>>> Masters - Computer Science
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Thanks & Regards
>>> Mehak Mehta
>>> Masters - Computer Science
>>>
>>
>>
>
>
> --
> Thanks & Regards
> Mehak Mehta
> Masters - Computer Science
>
Re: Nullpointer Exception in Kylin while checking the status of cube
Posted by Mehak Mehta <me...@stonybrook.edu>.
Hi Li,
Again thanks for your reply. I resolved that issue and I am able to build
the cube.
But after completion of building it. After building it my Cube size and
Source file is shown as 0.
As shown below.
[image: Inline image 1]
How can I check/debug this problem.
Can you help me with this.
I have checked logs and Kylin is retrieving data from Hive tables.
Thanks & Regards,
Mehak
On Mon, Dec 14, 2015 at 3:29 AM, Li Yang <li...@apache.org> wrote:
> Pls check https://issues.apache.org/jira/browse/KYLIN-953
>
>
> On Fri, Dec 11, 2015 at 8:25 PM, Mehak Mehta <me...@stonybrook.edu>
> wrote:
>
>> Hi,
>>
>> Thanks Li Yang for your reply.
>>
>> I solved that previous issue.
>> Now my cube is building but it always fails at 70% with the following
>> error:
>>
>> 2015-12-11 07:12:37,021 INFO [pool-4-thread-1]
>> mapreduce.HFileOutputFormat2: bulkload locality sensitive enabled
>> 2015-12-11 07:12:37,021 INFO [pool-4-thread-1]
>> mapreduce.HFileOutputFormat2: Looking up current regions for table
>> KYLIN_23ZG714AUU
>> 2015-12-11 07:12:37,029 INFO [pool-4-thread-1]
>> mapreduce.HFileOutputFormat2: Configuring 1 reduce partitions to match
>> current region count
>> [pool-4-thread-1]:[2015-12-11 07:12:37,030][ERROR]*[org.apache.kylin.job.hadoop.cube.CubeHFileJob.run(CubeHFileJob.java:95)]
>> - error in CubeHFileJob*
>> *java.lang.IllegalArgumentException: Can not create a Path from a null
>> string*
>> * at org.apache.hadoop.fs.Path.checkPathArg(Path.java:123)*
>> * at org.apache.hadoop.fs.Path.<init>(Path.java:135)*
>> * at org.apache.hadoop.fs.Path.<init>(Path.java:89)*
>> * at
>> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configurePartitioner(HFileOutputFormat2.java:615)*
>> * at
>> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad(HFileOutputFormat2.java:464)*
>> * at
>> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.configureIncrementalLoad(HFileOutputFormat.java:88)*
>> * at
>> org.apache.kylin.job.hadoop.cube.CubeHFileJob.run(CubeHFileJob.java:86)*
>> * at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)*
>> * at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)*
>> * at
>> org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:113)*
>> * at
>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
>> * at
>> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)*
>> * at
>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
>> * at
>> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)*
>> * at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)*
>> * at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)*
>> * at java.lang.Thread.run(Thread.java:745)*
>> *usage: CubeHFileJob*
>> * -cubename <name> Cube name. For exmaple, flat_item_cube*
>> * -htablename <htable name> HTable name*
>> * -input <path> Input path*
>> * -jobname <name> Job name. For exmaple,*
>> * Kylin_Cuboid_Builder-clsfd_v2_Step_22-D)*
>> -output <path> Output path
>> [pool-4-thread-1]:[2015-12-11
>> 07:12:37,034][INFO][org.apache.kylin.job.hadoop.AbstractHadoopJob.cleanupTempConfFile(AbstractHadoopJob.java:340)]
>> - tempMetaFileString is :
>> file:///data/shared/mehak/hadoop/kylin/./bin/../tomcat/temp/kylin_job_meta4467707123004651766/meta
>> *[pool-4-thread-1]:[2015-12-11
>> 07:12:37,040][ERROR][org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:116)]
>> - error execute
>> MapReduceExecutable{id=dbf394d0-5a18-4299-b191-c7bc0dc9b941-09,
>> name=Convert Cuboid Data to HFile, state=RUNNING}*
>> *java.lang.IllegalArgumentException: Can not create a Path from a null
>> string*
>> * at org.apache.hadoop.fs.Path.checkPathArg(Path.java:123)*
>> * at org.apache.hadoop.fs.Path.<init>(Path.java:135)*
>> * at org.apache.hadoop.fs.Path.<init>(Path.java:89)*
>> * at
>> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configurePartitioner(HFileOutputFormat2.java:615)*
>> * at
>> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad(HFileOutputFormat2.java:464)*
>> * at
>> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.configureIncrementalLoad(HFileOutputFormat.java:88)*
>> * at
>> org.apache.kylin.job.hadoop.cube.CubeHFileJob.run(CubeHFileJob.java:86)*
>> * at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)*
>> * at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)*
>> * at
>> org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:113)*
>> * at
>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
>> * at
>> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)*
>> * at
>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
>> * at
>> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)*
>> * at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)*
>> * at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)*
>> * at java.lang.Thread.run(Thread.java:745)*
>> [pool-4-thread-1]:[2015-12-11
>> 07:12:37,045][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>> - Saving resource /execute_output/dbf394d0-5a18-4299-b191-c7bc0dc9b941-09
>> (Store kylin_metadata@hbase)
>> [pool-4-thread-1]:[2015-12-11
>> 07:12:37,051][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>> - Saving resource /execute_output/dbf394d0-5a18-4299-b191-c7bc0dc9b941-09
>> (Store kylin_metadata@hbase)
>> [pool-4-thread-1]:[2015-12-11
>> 07:12:37,053][INFO][org.apache.kylin.job.manager.ExecutableManager.updateJobOutput(ExecutableManager.java:241)]
>> - job id:dbf394d0-5a18-4299-b191-c7bc0dc9b941-09 from RUNNING to ERROR
>>
>> I cannot see any errors in hadoop, hive or hbase logs. It seems to be
>> some configuration issue.
>> Kindly help, I have tried a lot configurations but it doesn't work.
>>
>> Thanks & Regards,
>> Mehak Mehta
>>
>>
>>
>> On Thu, Dec 10, 2015 at 4:42 AM, Li Yang <li...@apache.org> wrote:
>>
>>> Kylin depends on a hadoop config "yarn.resourcemanager.webapp.address"
>>> to find resource manager and come up a ws URL to query job status.
>>>
>>> You can check if the config presents in site xmls on Kylin classpath.
>>>
>>> Or if troubleshooting Hadoop is too painful, a simpler way is adding a
>>> line in conf/kylin.properties.
>>>
>>> kylin.job.yarn.app.rest.check.status.url=
>>> https://YOUR_RM_AND_PORT/ws/v1/cluster/apps/${job_id}?anonymous=true
>>>
>>> On Tue, Dec 8, 2015 at 11:46 AM, Mehak Mehta <mehak.mehta@stonybrook.edu
>>> > wrote:
>>>
>>>> Hi,
>>>>
>>>> I am getting the following error while building my cube using browser
>>>> console on my Hivetable.
>>>> I am using default architecture
>>>>
>>>> hadoop-2.6.0 (cluster of 10 nodes)
>>>> apache-hive-0.14.0 (using default metastore)
>>>> apache-kylin-1.1.1 (on single node)
>>>> hbase-0.98.16 (on single node as pseudo cluster)
>>>>
>>>> Note I have recompiled hadoop (with Snappy library) for my system
>>>> (Centos 6.5 64 bit).
>>>> It starts building the cube but fails while checking its status with
>>>> the following *Kylin logs,* can you please help me:
>>>>
>>>> 015-12-05 03:34:35,779 INFO [Thread-36] mapred.LocalJobRunner:
>>>> OutputCommitter is
>>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
>>>> 2015-12-05 03:34:35,848 INFO [Thread-36] mapred.LocalJobRunner:
>>>> Waiting for map tasks
>>>> 2015-12-05 03:34:35,849 INFO [LocalJobRunner Map Task Executor #0]
>>>> mapred.LocalJobRunner: Starting task:
>>>> attempt_local1931418873_0001_m_000000_0
>>>> 2015-12-05 03:34:35,890 INFO [LocalJobRunner Map Task Executor #0]
>>>> mapred.Task: Using ResourceCalculatorProcessTree : [ ]
>>>> 2015-12-05 03:34:35,897 INFO [LocalJobRunner Map Task Executor #0]
>>>> mapred.MapTask: Processing split:
>>>> org.apache.hive.hcatalog.mapreduce.HCatSplit@7d6447d
>>>> 2015-12-05 03:34:35,921 INFO [LocalJobRunner Map Task Executor #0]
>>>> mapred.MapTask: Map output collector class =
>>>> org.apache.hadoop.mapred.MapTask$MapOutputBuffer
>>>> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
>>>> mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
>>>> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
>>>> mapred.MapTask: mapreduce.task.io.sort.mb: 100
>>>> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
>>>> mapred.MapTask: soft limit at 83886080
>>>> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
>>>> mapred.MapTask: bufstart = 0; bufvoid = 104857600
>>>> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
>>>> mapred.MapTask: kvstart = 26214396; length = 6553600
>>>> [pool-4-thread-2]:[2015-12-05
>>>> 03:34:35,966][WARN][org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:682)]
>>>> - Going to buffer response body of large or unknown size. Using
>>>> getResponseBodyAsStream instead is recommended.
>>>> [pool-4-thread-2]:[2015-12-05
>>>> 03:34:35,967][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:92)]
>>>> - Job job_local1931418873_0001 get status check result.
>>>>
>>>> [pool-4-thread-2]:[2015-12-05
>>>> 03:34:35,994][ERROR][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:91)]
>>>> - error check status
>>>> *java.lang.NullPointerException*
>>>> * at
>>>> org.apache.kylin.job.tools.HadoopStatusGetter.get(HadoopStatusGetter.java:56)*
>>>> * at
>>>> org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:56)*
>>>> * at
>>>> org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:137)*
>>>> * at
>>>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
>>>> * at
>>>> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)*
>>>> * at
>>>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
>>>> * at
>>>> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)*
>>>> * at
>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)*
>>>> * at
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)*
>>>> * at java.lang.Thread.run(Thread.java:745)*
>>>> [pool-4-thread-2]:[2015-12-05
>>>> 03:34:35,998][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>>>> - Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
>>>> (Store kylin_metadata@hbase)
>>>> [pool-4-thread-2]:[2015-12-05
>>>> 03:34:36,003][DEBUG][org.apache.kylin.job.common.HadoopCmdOutput.updateJobCounter(HadoopCmdOutput.java:94)]
>>>> - Counters: 0
>>>> 2015-12-05 03:34:36,010 WARN [pool-4-thread-2] mapreduce.Counters:
>>>> Group FileSystemCounters is deprecated. Use
>>>> org.apache.hadoop.mapreduce.FileSystemCounter instead
>>>> [pool-4-thread-2]:[2015-12-05
>>>> 03:34:36,013][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>>>> - Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
>>>> (Store kylin_metadata@hbase)
>>>> [pool-4-thread-2]:[2015-12-05
>>>> 03:34:36,018][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>>>> - Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
>>>> (Store kylin_metadata@hbase)
>>>> [pool-4-thread-2]:[2015-12-05
>>>> 03:34:36,023][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>>>> - Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
>>>> (Store kylin_metadata@hbase)
>>>>
>>>>
>>>>
>>>> I couldn't see any error in my Hive logs or Hadoop logs, but there were
>>>> some logs related to this in *Hbase Logs:*
>>>>
>>>> 2015-12-05 03:29:43,208 DEBUG [region-location-2]
>>>> regionserver.HRegionFileSystem: No StoreFiles for:
>>>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/a
>>>> *2015-12-05 03:29:43,208 DEBUG [region-location-2]
>>>> regionserver.HRegionFileSystem: No StoreFiles for:
>>>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/q*
>>>> *2015-12-05 03:30:01,382 INFO
>>>> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181 <http://0.0.0.0/0.0.0.0:2181>]
>>>> server.NIOServerCnxnFactory: Accepted socket connection from
>>>> /127.0.0.1:36859 <http://127.0.0.1:36859/>*
>>>> *2015-12-05 03:30:01,387 INFO
>>>> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181 <http://0.0.0.0/0.0.0.0:2181>]
>>>> server.ZooKeeperServer: Client attempting to establish new session at
>>>> /127.0.0.1:36859 <http://127.0.0.1:36859/>*
>>>> *2015-12-05 03:30:01,411 INFO [SyncThread:0] server.ZooKeeperServer:
>>>> Established session 0x15171398d090008 with negotiated timeout 40000 for
>>>> client /127.0.0.1:36859 <http://127.0.0.1:36859/>*
>>>> *2015-12-05 03:30:03,761 INFO
>>>> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181 <http://0.0.0.0/0.0.0.0:2181>]
>>>> server.NIOServerCnxnFactory: Accepted socket connection from
>>>> /127.0.0.1:36862 <http://127.0.0.1:36862/>*
>>>> *2015-12-05 03:30:03,761 INFO
>>>> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181 <http://0.0.0.0/0.0.0.0:2181>]
>>>> server.ZooKeeperServer: Client attempting to establish new session at
>>>> /127.0.0.1:36862 <http://127.0.0.1:36862/>*
>>>> *2015-12-05 03:30:03,763 INFO [SyncThread:0] server.ZooKeeperServer:
>>>> Established session 0x15171398d090009 with negotiated timeout 40000 for
>>>> client /127.0.0.1:36862 <http://127.0.0.1:36862/>*
>>>> *2015-12-05 03:30:03,816 INFO [ProcessThread(sid:0 cport:-1):]
>>>> server.PrepRequestProcessor: Got user-level KeeperException when processing
>>>> sessionid:0x15171398d090009 type:create cxid:0x1 zxid:0x52d txntype:-1
>>>> reqpath:n/a Error Path:/kylin/job_engine/lock/kylin_metadata
>>>> Error:KeeperErrorCode = NoNode for /kylin/job_engine/lock/kylin_metadata*
>>>> *2015-12-05 03:30:43,210 DEBUG [region-location-3]
>>>> regionserver.HRegionFileSystem: No StoreFiles for:
>>>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/a*
>>>> *2015-12-05 03:30:43,211 DEBUG [region-location-3]
>>>> regionserver.HRegionFileSystem: No StoreFiles for:
>>>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/q*
>>>> *2015-12-05 03:31:43,210 DEBUG [region-location-4]
>>>> regionserver.HRegionFileSystem: No StoreFiles for:
>>>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/a*
>>>> *2015-12-05 03:31:43,210 DEBUG [region-location-4]
>>>> regionserver.HRegionFileSystem: No StoreFiles for:
>>>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/q*
>>>>
>>>> But I couldn't figure out the cause of the problem, can anybody please
>>>> help me in resolving this.
>>>> I have been struggling with this for sometime now.
>>>>
>>>> --
>>>> Thanks & Regards
>>>> Mehak Mehta
>>>> Masters - Computer Science
>>>>
>>>
>>>
>>
>>
>> --
>> Thanks & Regards
>> Mehak Mehta
>> Masters - Computer Science
>>
>
>
--
Thanks & Regards
Mehak Mehta
Masters - Computer Science
Re: Nullpointer Exception in Kylin while checking the status of cube
Posted by Li Yang <li...@apache.org>.
Pls check https://issues.apache.org/jira/browse/KYLIN-953
On Fri, Dec 11, 2015 at 8:25 PM, Mehak Mehta <me...@stonybrook.edu>
wrote:
> Hi,
>
> Thanks Li Yang for your reply.
>
> I solved that previous issue.
> Now my cube is building but it always fails at 70% with the following
> error:
>
> 2015-12-11 07:12:37,021 INFO [pool-4-thread-1]
> mapreduce.HFileOutputFormat2: bulkload locality sensitive enabled
> 2015-12-11 07:12:37,021 INFO [pool-4-thread-1]
> mapreduce.HFileOutputFormat2: Looking up current regions for table
> KYLIN_23ZG714AUU
> 2015-12-11 07:12:37,029 INFO [pool-4-thread-1]
> mapreduce.HFileOutputFormat2: Configuring 1 reduce partitions to match
> current region count
> [pool-4-thread-1]:[2015-12-11 07:12:37,030][ERROR]*[org.apache.kylin.job.hadoop.cube.CubeHFileJob.run(CubeHFileJob.java:95)]
> - error in CubeHFileJob*
> *java.lang.IllegalArgumentException: Can not create a Path from a null
> string*
> * at org.apache.hadoop.fs.Path.checkPathArg(Path.java:123)*
> * at org.apache.hadoop.fs.Path.<init>(Path.java:135)*
> * at org.apache.hadoop.fs.Path.<init>(Path.java:89)*
> * at
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configurePartitioner(HFileOutputFormat2.java:615)*
> * at
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad(HFileOutputFormat2.java:464)*
> * at
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.configureIncrementalLoad(HFileOutputFormat.java:88)*
> * at
> org.apache.kylin.job.hadoop.cube.CubeHFileJob.run(CubeHFileJob.java:86)*
> * at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)*
> * at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)*
> * at
> org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:113)*
> * at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
> * at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)*
> * at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
> * at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)*
> * at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)*
> * at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)*
> * at java.lang.Thread.run(Thread.java:745)*
> *usage: CubeHFileJob*
> * -cubename <name> Cube name. For exmaple, flat_item_cube*
> * -htablename <htable name> HTable name*
> * -input <path> Input path*
> * -jobname <name> Job name. For exmaple,*
> * Kylin_Cuboid_Builder-clsfd_v2_Step_22-D)*
> -output <path> Output path
> [pool-4-thread-1]:[2015-12-11
> 07:12:37,034][INFO][org.apache.kylin.job.hadoop.AbstractHadoopJob.cleanupTempConfFile(AbstractHadoopJob.java:340)]
> - tempMetaFileString is :
> file:///data/shared/mehak/hadoop/kylin/./bin/../tomcat/temp/kylin_job_meta4467707123004651766/meta
> *[pool-4-thread-1]:[2015-12-11
> 07:12:37,040][ERROR][org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:116)]
> - error execute
> MapReduceExecutable{id=dbf394d0-5a18-4299-b191-c7bc0dc9b941-09,
> name=Convert Cuboid Data to HFile, state=RUNNING}*
> *java.lang.IllegalArgumentException: Can not create a Path from a null
> string*
> * at org.apache.hadoop.fs.Path.checkPathArg(Path.java:123)*
> * at org.apache.hadoop.fs.Path.<init>(Path.java:135)*
> * at org.apache.hadoop.fs.Path.<init>(Path.java:89)*
> * at
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configurePartitioner(HFileOutputFormat2.java:615)*
> * at
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad(HFileOutputFormat2.java:464)*
> * at
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.configureIncrementalLoad(HFileOutputFormat.java:88)*
> * at
> org.apache.kylin.job.hadoop.cube.CubeHFileJob.run(CubeHFileJob.java:86)*
> * at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)*
> * at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)*
> * at
> org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:113)*
> * at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
> * at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)*
> * at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
> * at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)*
> * at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)*
> * at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)*
> * at java.lang.Thread.run(Thread.java:745)*
> [pool-4-thread-1]:[2015-12-11
> 07:12:37,045][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
> - Saving resource /execute_output/dbf394d0-5a18-4299-b191-c7bc0dc9b941-09
> (Store kylin_metadata@hbase)
> [pool-4-thread-1]:[2015-12-11
> 07:12:37,051][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
> - Saving resource /execute_output/dbf394d0-5a18-4299-b191-c7bc0dc9b941-09
> (Store kylin_metadata@hbase)
> [pool-4-thread-1]:[2015-12-11
> 07:12:37,053][INFO][org.apache.kylin.job.manager.ExecutableManager.updateJobOutput(ExecutableManager.java:241)]
> - job id:dbf394d0-5a18-4299-b191-c7bc0dc9b941-09 from RUNNING to ERROR
>
> I cannot see any errors in hadoop, hive or hbase logs. It seems to be some
> configuration issue.
> Kindly help, I have tried a lot configurations but it doesn't work.
>
> Thanks & Regards,
> Mehak Mehta
>
>
>
> On Thu, Dec 10, 2015 at 4:42 AM, Li Yang <li...@apache.org> wrote:
>
>> Kylin depends on a hadoop config "yarn.resourcemanager.webapp.address" to
>> find resource manager and come up a ws URL to query job status.
>>
>> You can check if the config presents in site xmls on Kylin classpath.
>>
>> Or if troubleshooting Hadoop is too painful, a simpler way is adding a
>> line in conf/kylin.properties.
>>
>> kylin.job.yarn.app.rest.check.status.url=
>> https://YOUR_RM_AND_PORT/ws/v1/cluster/apps/${job_id}?anonymous=true
>>
>> On Tue, Dec 8, 2015 at 11:46 AM, Mehak Mehta <me...@stonybrook.edu>
>> wrote:
>>
>>> Hi,
>>>
>>> I am getting the following error while building my cube using browser
>>> console on my Hivetable.
>>> I am using default architecture
>>>
>>> hadoop-2.6.0 (cluster of 10 nodes)
>>> apache-hive-0.14.0 (using default metastore)
>>> apache-kylin-1.1.1 (on single node)
>>> hbase-0.98.16 (on single node as pseudo cluster)
>>>
>>> Note I have recompiled hadoop (with Snappy library) for my system
>>> (Centos 6.5 64 bit).
>>> It starts building the cube but fails while checking its status with the
>>> following *Kylin logs,* can you please help me:
>>>
>>> 015-12-05 03:34:35,779 INFO [Thread-36] mapred.LocalJobRunner:
>>> OutputCommitter is
>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
>>> 2015-12-05 03:34:35,848 INFO [Thread-36] mapred.LocalJobRunner: Waiting
>>> for map tasks
>>> 2015-12-05 03:34:35,849 INFO [LocalJobRunner Map Task Executor #0]
>>> mapred.LocalJobRunner: Starting task:
>>> attempt_local1931418873_0001_m_000000_0
>>> 2015-12-05 03:34:35,890 INFO [LocalJobRunner Map Task Executor #0]
>>> mapred.Task: Using ResourceCalculatorProcessTree : [ ]
>>> 2015-12-05 03:34:35,897 INFO [LocalJobRunner Map Task Executor #0]
>>> mapred.MapTask: Processing split:
>>> org.apache.hive.hcatalog.mapreduce.HCatSplit@7d6447d
>>> 2015-12-05 03:34:35,921 INFO [LocalJobRunner Map Task Executor #0]
>>> mapred.MapTask: Map output collector class =
>>> org.apache.hadoop.mapred.MapTask$MapOutputBuffer
>>> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
>>> mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
>>> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
>>> mapred.MapTask: mapreduce.task.io.sort.mb: 100
>>> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
>>> mapred.MapTask: soft limit at 83886080
>>> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
>>> mapred.MapTask: bufstart = 0; bufvoid = 104857600
>>> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
>>> mapred.MapTask: kvstart = 26214396; length = 6553600
>>> [pool-4-thread-2]:[2015-12-05
>>> 03:34:35,966][WARN][org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:682)]
>>> - Going to buffer response body of large or unknown size. Using
>>> getResponseBodyAsStream instead is recommended.
>>> [pool-4-thread-2]:[2015-12-05
>>> 03:34:35,967][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:92)]
>>> - Job job_local1931418873_0001 get status check result.
>>>
>>> [pool-4-thread-2]:[2015-12-05
>>> 03:34:35,994][ERROR][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:91)]
>>> - error check status
>>> *java.lang.NullPointerException*
>>> * at
>>> org.apache.kylin.job.tools.HadoopStatusGetter.get(HadoopStatusGetter.java:56)*
>>> * at
>>> org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:56)*
>>> * at
>>> org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:137)*
>>> * at
>>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
>>> * at
>>> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)*
>>> * at
>>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
>>> * at
>>> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)*
>>> * at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)*
>>> * at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)*
>>> * at java.lang.Thread.run(Thread.java:745)*
>>> [pool-4-thread-2]:[2015-12-05
>>> 03:34:35,998][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>>> - Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
>>> (Store kylin_metadata@hbase)
>>> [pool-4-thread-2]:[2015-12-05
>>> 03:34:36,003][DEBUG][org.apache.kylin.job.common.HadoopCmdOutput.updateJobCounter(HadoopCmdOutput.java:94)]
>>> - Counters: 0
>>> 2015-12-05 03:34:36,010 WARN [pool-4-thread-2] mapreduce.Counters:
>>> Group FileSystemCounters is deprecated. Use
>>> org.apache.hadoop.mapreduce.FileSystemCounter instead
>>> [pool-4-thread-2]:[2015-12-05
>>> 03:34:36,013][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>>> - Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
>>> (Store kylin_metadata@hbase)
>>> [pool-4-thread-2]:[2015-12-05
>>> 03:34:36,018][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>>> - Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
>>> (Store kylin_metadata@hbase)
>>> [pool-4-thread-2]:[2015-12-05
>>> 03:34:36,023][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>>> - Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
>>> (Store kylin_metadata@hbase)
>>>
>>>
>>>
>>> I couldn't see any error in my Hive logs or Hadoop logs, but there were
>>> some logs related to this in *Hbase Logs:*
>>>
>>> 2015-12-05 03:29:43,208 DEBUG [region-location-2]
>>> regionserver.HRegionFileSystem: No StoreFiles for:
>>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/a
>>> *2015-12-05 03:29:43,208 DEBUG [region-location-2]
>>> regionserver.HRegionFileSystem: No StoreFiles for:
>>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/q*
>>> *2015-12-05 03:30:01,382 INFO
>>> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181 <http://0.0.0.0/0.0.0.0:2181>]
>>> server.NIOServerCnxnFactory: Accepted socket connection from
>>> /127.0.0.1:36859 <http://127.0.0.1:36859/>*
>>> *2015-12-05 03:30:01,387 INFO
>>> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181 <http://0.0.0.0/0.0.0.0:2181>]
>>> server.ZooKeeperServer: Client attempting to establish new session at
>>> /127.0.0.1:36859 <http://127.0.0.1:36859/>*
>>> *2015-12-05 03:30:01,411 INFO [SyncThread:0] server.ZooKeeperServer:
>>> Established session 0x15171398d090008 with negotiated timeout 40000 for
>>> client /127.0.0.1:36859 <http://127.0.0.1:36859/>*
>>> *2015-12-05 03:30:03,761 INFO
>>> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181 <http://0.0.0.0/0.0.0.0:2181>]
>>> server.NIOServerCnxnFactory: Accepted socket connection from
>>> /127.0.0.1:36862 <http://127.0.0.1:36862/>*
>>> *2015-12-05 03:30:03,761 INFO
>>> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181 <http://0.0.0.0/0.0.0.0:2181>]
>>> server.ZooKeeperServer: Client attempting to establish new session at
>>> /127.0.0.1:36862 <http://127.0.0.1:36862/>*
>>> *2015-12-05 03:30:03,763 INFO [SyncThread:0] server.ZooKeeperServer:
>>> Established session 0x15171398d090009 with negotiated timeout 40000 for
>>> client /127.0.0.1:36862 <http://127.0.0.1:36862/>*
>>> *2015-12-05 03:30:03,816 INFO [ProcessThread(sid:0 cport:-1):]
>>> server.PrepRequestProcessor: Got user-level KeeperException when processing
>>> sessionid:0x15171398d090009 type:create cxid:0x1 zxid:0x52d txntype:-1
>>> reqpath:n/a Error Path:/kylin/job_engine/lock/kylin_metadata
>>> Error:KeeperErrorCode = NoNode for /kylin/job_engine/lock/kylin_metadata*
>>> *2015-12-05 03:30:43,210 DEBUG [region-location-3]
>>> regionserver.HRegionFileSystem: No StoreFiles for:
>>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/a*
>>> *2015-12-05 03:30:43,211 DEBUG [region-location-3]
>>> regionserver.HRegionFileSystem: No StoreFiles for:
>>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/q*
>>> *2015-12-05 03:31:43,210 DEBUG [region-location-4]
>>> regionserver.HRegionFileSystem: No StoreFiles for:
>>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/a*
>>> *2015-12-05 03:31:43,210 DEBUG [region-location-4]
>>> regionserver.HRegionFileSystem: No StoreFiles for:
>>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/q*
>>>
>>> But I couldn't figure out the cause of the problem, can anybody please
>>> help me in resolving this.
>>> I have been struggling with this for sometime now.
>>>
>>> --
>>> Thanks & Regards
>>> Mehak Mehta
>>> Masters - Computer Science
>>>
>>
>>
>
>
> --
> Thanks & Regards
> Mehak Mehta
> Masters - Computer Science
>
Re: Nullpointer Exception in Kylin while checking the status of cube
Posted by Mehak Mehta <me...@stonybrook.edu>.
Hi,
Thanks Li Yang for your reply.
I solved that previous issue.
Now my cube is building but it always fails at 70% with the following error:
2015-12-11 07:12:37,021 INFO [pool-4-thread-1]
mapreduce.HFileOutputFormat2: bulkload locality sensitive enabled
2015-12-11 07:12:37,021 INFO [pool-4-thread-1]
mapreduce.HFileOutputFormat2: Looking up current regions for table
KYLIN_23ZG714AUU
2015-12-11 07:12:37,029 INFO [pool-4-thread-1]
mapreduce.HFileOutputFormat2: Configuring 1 reduce partitions to match
current region count
[pool-4-thread-1]:[2015-12-11
07:12:37,030][ERROR]*[org.apache.kylin.job.hadoop.cube.CubeHFileJob.run(CubeHFileJob.java:95)]
- error in CubeHFileJob*
*java.lang.IllegalArgumentException: Can not create a Path from a null
string*
* at org.apache.hadoop.fs.Path.checkPathArg(Path.java:123)*
* at org.apache.hadoop.fs.Path.<init>(Path.java:135)*
* at org.apache.hadoop.fs.Path.<init>(Path.java:89)*
* at
org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configurePartitioner(HFileOutputFormat2.java:615)*
* at
org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad(HFileOutputFormat2.java:464)*
* at
org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.configureIncrementalLoad(HFileOutputFormat.java:88)*
* at
org.apache.kylin.job.hadoop.cube.CubeHFileJob.run(CubeHFileJob.java:86)*
* at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)*
* at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)*
* at
org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:113)*
* at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
* at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)*
* at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
* at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)*
* at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)*
* at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)*
* at java.lang.Thread.run(Thread.java:745)*
*usage: CubeHFileJob*
* -cubename <name> Cube name. For exmaple, flat_item_cube*
* -htablename <htable name> HTable name*
* -input <path> Input path*
* -jobname <name> Job name. For exmaple,*
* Kylin_Cuboid_Builder-clsfd_v2_Step_22-D)*
-output <path> Output path
[pool-4-thread-1]:[2015-12-11
07:12:37,034][INFO][org.apache.kylin.job.hadoop.AbstractHadoopJob.cleanupTempConfFile(AbstractHadoopJob.java:340)]
- tempMetaFileString is :
file:///data/shared/mehak/hadoop/kylin/./bin/../tomcat/temp/kylin_job_meta4467707123004651766/meta
*[pool-4-thread-1]:[2015-12-11
07:12:37,040][ERROR][org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:116)]
- error execute
MapReduceExecutable{id=dbf394d0-5a18-4299-b191-c7bc0dc9b941-09,
name=Convert Cuboid Data to HFile, state=RUNNING}*
*java.lang.IllegalArgumentException: Can not create a Path from a null
string*
* at org.apache.hadoop.fs.Path.checkPathArg(Path.java:123)*
* at org.apache.hadoop.fs.Path.<init>(Path.java:135)*
* at org.apache.hadoop.fs.Path.<init>(Path.java:89)*
* at
org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configurePartitioner(HFileOutputFormat2.java:615)*
* at
org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad(HFileOutputFormat2.java:464)*
* at
org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.configureIncrementalLoad(HFileOutputFormat.java:88)*
* at
org.apache.kylin.job.hadoop.cube.CubeHFileJob.run(CubeHFileJob.java:86)*
* at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)*
* at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)*
* at
org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:113)*
* at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
* at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)*
* at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
* at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)*
* at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)*
* at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)*
* at java.lang.Thread.run(Thread.java:745)*
[pool-4-thread-1]:[2015-12-11
07:12:37,045][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
- Saving resource /execute_output/dbf394d0-5a18-4299-b191-c7bc0dc9b941-09
(Store kylin_metadata@hbase)
[pool-4-thread-1]:[2015-12-11
07:12:37,051][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
- Saving resource /execute_output/dbf394d0-5a18-4299-b191-c7bc0dc9b941-09
(Store kylin_metadata@hbase)
[pool-4-thread-1]:[2015-12-11
07:12:37,053][INFO][org.apache.kylin.job.manager.ExecutableManager.updateJobOutput(ExecutableManager.java:241)]
- job id:dbf394d0-5a18-4299-b191-c7bc0dc9b941-09 from RUNNING to ERROR
I cannot see any errors in hadoop, hive or hbase logs. It seems to be some
configuration issue.
Kindly help, I have tried a lot configurations but it doesn't work.
Thanks & Regards,
Mehak Mehta
On Thu, Dec 10, 2015 at 4:42 AM, Li Yang <li...@apache.org> wrote:
> Kylin depends on a hadoop config "yarn.resourcemanager.webapp.address" to
> find resource manager and come up a ws URL to query job status.
>
> You can check if the config presents in site xmls on Kylin classpath.
>
> Or if troubleshooting Hadoop is too painful, a simpler way is adding a
> line in conf/kylin.properties.
>
> kylin.job.yarn.app.rest.check.status.url=
> https://YOUR_RM_AND_PORT/ws/v1/cluster/apps/${job_id}?anonymous=true
>
> On Tue, Dec 8, 2015 at 11:46 AM, Mehak Mehta <me...@stonybrook.edu>
> wrote:
>
>> Hi,
>>
>> I am getting the following error while building my cube using browser
>> console on my Hivetable.
>> I am using default architecture
>>
>> hadoop-2.6.0 (cluster of 10 nodes)
>> apache-hive-0.14.0 (using default metastore)
>> apache-kylin-1.1.1 (on single node)
>> hbase-0.98.16 (on single node as pseudo cluster)
>>
>> Note I have recompiled hadoop (with Snappy library) for my system (Centos
>> 6.5 64 bit).
>> It starts building the cube but fails while checking its status with the
>> following *Kylin logs,* can you please help me:
>>
>> 015-12-05 03:34:35,779 INFO [Thread-36] mapred.LocalJobRunner:
>> OutputCommitter is
>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
>> 2015-12-05 03:34:35,848 INFO [Thread-36] mapred.LocalJobRunner: Waiting
>> for map tasks
>> 2015-12-05 03:34:35,849 INFO [LocalJobRunner Map Task Executor #0]
>> mapred.LocalJobRunner: Starting task:
>> attempt_local1931418873_0001_m_000000_0
>> 2015-12-05 03:34:35,890 INFO [LocalJobRunner Map Task Executor #0]
>> mapred.Task: Using ResourceCalculatorProcessTree : [ ]
>> 2015-12-05 03:34:35,897 INFO [LocalJobRunner Map Task Executor #0]
>> mapred.MapTask: Processing split:
>> org.apache.hive.hcatalog.mapreduce.HCatSplit@7d6447d
>> 2015-12-05 03:34:35,921 INFO [LocalJobRunner Map Task Executor #0]
>> mapred.MapTask: Map output collector class =
>> org.apache.hadoop.mapred.MapTask$MapOutputBuffer
>> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
>> mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
>> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
>> mapred.MapTask: mapreduce.task.io.sort.mb: 100
>> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
>> mapred.MapTask: soft limit at 83886080
>> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
>> mapred.MapTask: bufstart = 0; bufvoid = 104857600
>> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
>> mapred.MapTask: kvstart = 26214396; length = 6553600
>> [pool-4-thread-2]:[2015-12-05
>> 03:34:35,966][WARN][org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:682)]
>> - Going to buffer response body of large or unknown size. Using
>> getResponseBodyAsStream instead is recommended.
>> [pool-4-thread-2]:[2015-12-05
>> 03:34:35,967][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:92)]
>> - Job job_local1931418873_0001 get status check result.
>>
>> [pool-4-thread-2]:[2015-12-05
>> 03:34:35,994][ERROR][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:91)]
>> - error check status
>> *java.lang.NullPointerException*
>> * at
>> org.apache.kylin.job.tools.HadoopStatusGetter.get(HadoopStatusGetter.java:56)*
>> * at
>> org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:56)*
>> * at
>> org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:137)*
>> * at
>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
>> * at
>> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)*
>> * at
>> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
>> * at
>> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)*
>> * at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)*
>> * at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)*
>> * at java.lang.Thread.run(Thread.java:745)*
>> [pool-4-thread-2]:[2015-12-05
>> 03:34:35,998][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>> - Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
>> (Store kylin_metadata@hbase)
>> [pool-4-thread-2]:[2015-12-05
>> 03:34:36,003][DEBUG][org.apache.kylin.job.common.HadoopCmdOutput.updateJobCounter(HadoopCmdOutput.java:94)]
>> - Counters: 0
>> 2015-12-05 03:34:36,010 WARN [pool-4-thread-2] mapreduce.Counters: Group
>> FileSystemCounters is deprecated. Use
>> org.apache.hadoop.mapreduce.FileSystemCounter instead
>> [pool-4-thread-2]:[2015-12-05
>> 03:34:36,013][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>> - Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
>> (Store kylin_metadata@hbase)
>> [pool-4-thread-2]:[2015-12-05
>> 03:34:36,018][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>> - Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
>> (Store kylin_metadata@hbase)
>> [pool-4-thread-2]:[2015-12-05
>> 03:34:36,023][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
>> - Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
>> (Store kylin_metadata@hbase)
>>
>>
>>
>> I couldn't see any error in my Hive logs or Hadoop logs, but there were
>> some logs related to this in *Hbase Logs:*
>>
>> 2015-12-05 03:29:43,208 DEBUG [region-location-2]
>> regionserver.HRegionFileSystem: No StoreFiles for:
>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/a
>> *2015-12-05 03:29:43,208 DEBUG [region-location-2]
>> regionserver.HRegionFileSystem: No StoreFiles for:
>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/q*
>> *2015-12-05 03:30:01,382 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181
>> <http://0.0.0.0/0.0.0.0:2181>] server.NIOServerCnxnFactory: Accepted socket
>> connection from /127.0.0.1:36859 <http://127.0.0.1:36859/>*
>> *2015-12-05 03:30:01,387 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181
>> <http://0.0.0.0/0.0.0.0:2181>] server.ZooKeeperServer: Client attempting to
>> establish new session at /127.0.0.1:36859 <http://127.0.0.1:36859/>*
>> *2015-12-05 03:30:01,411 INFO [SyncThread:0] server.ZooKeeperServer:
>> Established session 0x15171398d090008 with negotiated timeout 40000 for
>> client /127.0.0.1:36859 <http://127.0.0.1:36859/>*
>> *2015-12-05 03:30:03,761 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181
>> <http://0.0.0.0/0.0.0.0:2181>] server.NIOServerCnxnFactory: Accepted socket
>> connection from /127.0.0.1:36862 <http://127.0.0.1:36862/>*
>> *2015-12-05 03:30:03,761 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181
>> <http://0.0.0.0/0.0.0.0:2181>] server.ZooKeeperServer: Client attempting to
>> establish new session at /127.0.0.1:36862 <http://127.0.0.1:36862/>*
>> *2015-12-05 03:30:03,763 INFO [SyncThread:0] server.ZooKeeperServer:
>> Established session 0x15171398d090009 with negotiated timeout 40000 for
>> client /127.0.0.1:36862 <http://127.0.0.1:36862/>*
>> *2015-12-05 03:30:03,816 INFO [ProcessThread(sid:0 cport:-1):]
>> server.PrepRequestProcessor: Got user-level KeeperException when processing
>> sessionid:0x15171398d090009 type:create cxid:0x1 zxid:0x52d txntype:-1
>> reqpath:n/a Error Path:/kylin/job_engine/lock/kylin_metadata
>> Error:KeeperErrorCode = NoNode for /kylin/job_engine/lock/kylin_metadata*
>> *2015-12-05 03:30:43,210 DEBUG [region-location-3]
>> regionserver.HRegionFileSystem: No StoreFiles for:
>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/a*
>> *2015-12-05 03:30:43,211 DEBUG [region-location-3]
>> regionserver.HRegionFileSystem: No StoreFiles for:
>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/q*
>> *2015-12-05 03:31:43,210 DEBUG [region-location-4]
>> regionserver.HRegionFileSystem: No StoreFiles for:
>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/a*
>> *2015-12-05 03:31:43,210 DEBUG [region-location-4]
>> regionserver.HRegionFileSystem: No StoreFiles for:
>> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/q*
>>
>> But I couldn't figure out the cause of the problem, can anybody please
>> help me in resolving this.
>> I have been struggling with this for sometime now.
>>
>> --
>> Thanks & Regards
>> Mehak Mehta
>> Masters - Computer Science
>>
>
>
--
Thanks & Regards
Mehak Mehta
Masters - Computer Science
Re: Nullpointer Exception in Kylin while checking the status of cube
Posted by Li Yang <li...@apache.org>.
Kylin depends on a hadoop config "yarn.resourcemanager.webapp.address" to
find resource manager and come up a ws URL to query job status.
You can check if the config presents in site xmls on Kylin classpath.
Or if troubleshooting Hadoop is too painful, a simpler way is adding a line
in conf/kylin.properties.
kylin.job.yarn.app.rest.check.status.url=
https://YOUR_RM_AND_PORT/ws/v1/cluster/apps/${job_id}?anonymous=true
On Tue, Dec 8, 2015 at 11:46 AM, Mehak Mehta <me...@stonybrook.edu>
wrote:
> Hi,
>
> I am getting the following error while building my cube using browser
> console on my Hivetable.
> I am using default architecture
>
> hadoop-2.6.0 (cluster of 10 nodes)
> apache-hive-0.14.0 (using default metastore)
> apache-kylin-1.1.1 (on single node)
> hbase-0.98.16 (on single node as pseudo cluster)
>
> Note I have recompiled hadoop (with Snappy library) for my system (Centos
> 6.5 64 bit).
> It starts building the cube but fails while checking its status with the
> following *Kylin logs,* can you please help me:
>
> 015-12-05 03:34:35,779 INFO [Thread-36] mapred.LocalJobRunner:
> OutputCommitter is
> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
> 2015-12-05 03:34:35,848 INFO [Thread-36] mapred.LocalJobRunner: Waiting
> for map tasks
> 2015-12-05 03:34:35,849 INFO [LocalJobRunner Map Task Executor #0]
> mapred.LocalJobRunner: Starting task:
> attempt_local1931418873_0001_m_000000_0
> 2015-12-05 03:34:35,890 INFO [LocalJobRunner Map Task Executor #0]
> mapred.Task: Using ResourceCalculatorProcessTree : [ ]
> 2015-12-05 03:34:35,897 INFO [LocalJobRunner Map Task Executor #0]
> mapred.MapTask: Processing split:
> org.apache.hive.hcatalog.mapreduce.HCatSplit@7d6447d
> 2015-12-05 03:34:35,921 INFO [LocalJobRunner Map Task Executor #0]
> mapred.MapTask: Map output collector class =
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer
> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
> mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
> mapred.MapTask: mapreduce.task.io.sort.mb: 100
> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
> mapred.MapTask: soft limit at 83886080
> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
> mapred.MapTask: bufstart = 0; bufvoid = 104857600
> 2015-12-05 03:34:35,963 INFO [LocalJobRunner Map Task Executor #0]
> mapred.MapTask: kvstart = 26214396; length = 6553600
> [pool-4-thread-2]:[2015-12-05
> 03:34:35,966][WARN][org.apache.commons.httpclient.HttpMethodBase.getResponseBody(HttpMethodBase.java:682)]
> - Going to buffer response body of large or unknown size. Using
> getResponseBodyAsStream instead is recommended.
> [pool-4-thread-2]:[2015-12-05
> 03:34:35,967][DEBUG][org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:92)]
> - Job job_local1931418873_0001 get status check result.
>
> [pool-4-thread-2]:[2015-12-05
> 03:34:35,994][ERROR][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:91)]
> - error check status
> *java.lang.NullPointerException*
> * at
> org.apache.kylin.job.tools.HadoopStatusGetter.get(HadoopStatusGetter.java:56)*
> * at
> org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:56)*
> * at
> org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:137)*
> * at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
> * at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)*
> * at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)*
> * at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)*
> * at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)*
> * at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)*
> * at java.lang.Thread.run(Thread.java:745)*
> [pool-4-thread-2]:[2015-12-05
> 03:34:35,998][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
> - Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
> (Store kylin_metadata@hbase)
> [pool-4-thread-2]:[2015-12-05
> 03:34:36,003][DEBUG][org.apache.kylin.job.common.HadoopCmdOutput.updateJobCounter(HadoopCmdOutput.java:94)]
> - Counters: 0
> 2015-12-05 03:34:36,010 WARN [pool-4-thread-2] mapreduce.Counters: Group
> FileSystemCounters is deprecated. Use
> org.apache.hadoop.mapreduce.FileSystemCounter instead
> [pool-4-thread-2]:[2015-12-05
> 03:34:36,013][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
> - Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
> (Store kylin_metadata@hbase)
> [pool-4-thread-2]:[2015-12-05
> 03:34:36,018][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
> - Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
> (Store kylin_metadata@hbase)
> [pool-4-thread-2]:[2015-12-05
> 03:34:36,023][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:195)]
> - Saving resource /execute_output/d167963c-0bd0-422a-8abf-5c288d4f5131-01
> (Store kylin_metadata@hbase)
>
>
>
> I couldn't see any error in my Hive logs or Hadoop logs, but there were
> some logs related to this in *Hbase Logs:*
>
> 2015-12-05 03:29:43,208 DEBUG [region-location-2]
> regionserver.HRegionFileSystem: No StoreFiles for:
> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/a
> *2015-12-05 03:29:43,208 DEBUG [region-location-2]
> regionserver.HRegionFileSystem: No StoreFiles for:
> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/q*
> *2015-12-05 03:30:01,382 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181
> <http://0.0.0.0/0.0.0.0:2181>] server.NIOServerCnxnFactory: Accepted socket
> connection from /127.0.0.1:36859 <http://127.0.0.1:36859/>*
> *2015-12-05 03:30:01,387 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181
> <http://0.0.0.0/0.0.0.0:2181>] server.ZooKeeperServer: Client attempting to
> establish new session at /127.0.0.1:36859 <http://127.0.0.1:36859/>*
> *2015-12-05 03:30:01,411 INFO [SyncThread:0] server.ZooKeeperServer:
> Established session 0x15171398d090008 with negotiated timeout 40000 for
> client /127.0.0.1:36859 <http://127.0.0.1:36859/>*
> *2015-12-05 03:30:03,761 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181
> <http://0.0.0.0/0.0.0.0:2181>] server.NIOServerCnxnFactory: Accepted socket
> connection from /127.0.0.1:36862 <http://127.0.0.1:36862/>*
> *2015-12-05 03:30:03,761 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181
> <http://0.0.0.0/0.0.0.0:2181>] server.ZooKeeperServer: Client attempting to
> establish new session at /127.0.0.1:36862 <http://127.0.0.1:36862/>*
> *2015-12-05 03:30:03,763 INFO [SyncThread:0] server.ZooKeeperServer:
> Established session 0x15171398d090009 with negotiated timeout 40000 for
> client /127.0.0.1:36862 <http://127.0.0.1:36862/>*
> *2015-12-05 03:30:03,816 INFO [ProcessThread(sid:0 cport:-1):]
> server.PrepRequestProcessor: Got user-level KeeperException when processing
> sessionid:0x15171398d090009 type:create cxid:0x1 zxid:0x52d txntype:-1
> reqpath:n/a Error Path:/kylin/job_engine/lock/kylin_metadata
> Error:KeeperErrorCode = NoNode for /kylin/job_engine/lock/kylin_metadata*
> *2015-12-05 03:30:43,210 DEBUG [region-location-3]
> regionserver.HRegionFileSystem: No StoreFiles for:
> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/a*
> *2015-12-05 03:30:43,211 DEBUG [region-location-3]
> regionserver.HRegionFileSystem: No StoreFiles for:
> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/q*
> *2015-12-05 03:31:43,210 DEBUG [region-location-4]
> regionserver.HRegionFileSystem: No StoreFiles for:
> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/a*
> *2015-12-05 03:31:43,210 DEBUG [region-location-4]
> regionserver.HRegionFileSystem: No StoreFiles for:
> file:/data/shared/mehak/hadoop/hbase/hfiles/data/default/kylin_metadata_user/57de3063b02c8224546d5e67e5814455/q*
>
> But I couldn't figure out the cause of the problem, can anybody please
> help me in resolving this.
> I have been struggling with this for sometime now.
>
> --
> Thanks & Regards
> Mehak Mehta
> Masters - Computer Science
>