You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by "Luke Han (JIRA)" <ji...@apache.org> on 2015/08/14 09:54:46 UTC
[jira] [Updated] (KYLIN-404) Can't get cube source record size.
[ https://issues.apache.org/jira/browse/KYLIN-404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luke Han updated KYLIN-404:
---------------------------
Fix Version/s: v0.7.3
> Can't get cube source record size.
> ----------------------------------
>
> Key: KYLIN-404
> URL: https://issues.apache.org/jira/browse/KYLIN-404
> Project: Kylin
> Issue Type: Bug
> Reporter: Luke Han
> Assignee: Shaofeng SHI
> Labels: github-import
> Fix For: v0.7.3
>
>
> Dear All!
> I deploy kylin on Apache Hadoop, when i build the job , it failed on the last step. here is the some log message:
> ```java
> [QuartzScheduler_Worker-4]:[2014-11-24 19:24:47,280][INFO][com.kylinolap.job.flow.JobFlowListener.jobWasExecuted(JobFlowListener.java:93)] - cube_job_group.test_kylin_cube_without_slr_empty.30bbd69c-33ae-4983-aaf7-f46ab42b88dd.14 status: FINISHED
> [QuartzScheduler_Worker-4]:[2014-11-24 19:24:47,280][DEBUG][com.kylinolap.cube.CubeManager.loadCubeInstance(CubeManager.java:604)] - Loading CubeInstance kylin_metadata_qa(key='/cube/test_kylin_cube_without_slr_empty.json')@kylin_metadata_qa@hbase:192.168.44.21:2181:/hbase9827
> [QuartzScheduler_Worker-4]:[2014-11-24 19:24:47,295][DEBUG][com.kylinolap.common.persistence.ResourceStore.putResource(ResourceStore.java:166)] - Saving resource /job/30bbd69c-33ae-4983-aaf7-f46ab42b88dd (Store kylin_metadata_qa@hbase:192.168.44.21:2181:/hbase9827)
> [QuartzScheduler_Worker-4]:[2014-11-24 19:24:47,300][INFO][com.kylinolap.job.flow.JobFlowListener.updateCubeSegmentInfoOnSucceed(JobFlowListener.java:238)] - Updating cube segment FULL_BUILD for cube test_kylin_cube_without_slr_empty
> [QuartzScheduler_Worker-4]:[2014-11-24 19:24:47,301][ERROR][com.kylinolap.job.flow.JobFlowListener.jobWasExecuted(JobFlowListener.java:117)] - Can't get cube source record size.
> java.lang.RuntimeException: Can't get cube source record size.
> at com.kylinolap.job.flow.JobFlowListener.updateCubeSegmentInfoOnSucceed(JobFlowListener.java:274)
> at com.kylinolap.job.flow.JobFlowListener.jobWasExecuted(JobFlowListener.java:99)
> at org.quartz.core.QuartzScheduler.notifyJobListenersWasExecuted(QuartzScheduler.java:1985)
> at org.quartz.core.JobRunShell.notifyJobListenersComplete(JobRunShell.java:340)
> at org.quartz.core.JobRunShell.run(JobRunShell.java:224)
> at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
> ```
> here is some version info:
> Kylin 0.6.2
> hadoop 2.4.1
> hbase 0.98.4-hadoop2
> zookeeper 3.4.5
> hive 0.13.0
> I find some config info on hbase
> ```
> get 'kylin_metadata_qa_job', '/job/1eaf05f8-e9f5-4dd7-9748-ab9b3098a512'
> {
> "uuid": "1eaf05f8-e9f5-4dd7-9748-ab9b3098a512",
> "name": "test_kylin_cube_without_slr_empty - FULL_BUILD - BUILD - PST 2014-11-24 23:00:15",
> "type": "BUILD",
> "duration": 461,
> "steps": [
> {
> "name": "Create Intermediate Flat Hive Table",
> "info": null,
> "interruptCmd": null,
> "sequence_id": 0,
> "exec_cmd": "hive -e "DROPTABLEIFEXISTSkylin_intermediate_test_kylin_cube_without_slr_desc_FULL_BUILD_1eaf05f8_e9f5_4dd7_9748_ab9b3098a512;nCREATEEXTERNALTABLEIFNOTEXISTSkylin_intermediate_test_kylin_cube_without_slr_desc_FULL_BUILD_1eaf05f8_e9f5_4dd7_9748_ab9b3098a512n(nCAL_DTdaten,
> LEAF_CATEG_IDintn,
> LSTG_SITE_IDintn,
> META_CATEG_NAMEstringn,
> CATEG_LVL2_NAMEstringn,
> CATEG_LVL3_NAMEstringn,
> LSTG_FORMAT_NAMEstringn,
> SLR_SEGMENT_CDsmallintn,
> PRICEdecimal(38,
> 16)n,
> SELLER_IDbigintn)
> ...}
> ]
> ```
> in source code: com.kylinolap.job.flow.JobFlowListener.updateCubeSegmentInfoOnSucceed(JobFlowListener.java:272)
> ```
> JobStep createFlatTableStep = jobInstance.findStep(JobConstants.STEP_NAME_CREATE_FLAT_HIVE_TABLE);
> if (null != createFlatTableStep) {
> String sourceRecordsSize = createFlatTableStep.getInfo(JobInstance.SOURCE_RECORDS_SIZE);
> if (sourceRecordsSize == null || sourceRecordsSize.equals("")) {
> throw new RuntimeException("Can't get cube source record size.");
> }
> sourceSize = Long.parseLong(sourceRecordsSize);
> } else {
> log.info("No step with name '" + JobConstants.STEP_NAME_CREATE_FLAT_HIVE_TABLE + "' is found");
> }
> ```
> the "info" is null, cased throw a RuntimeException, some one have idea with it??
> ---------------- Imported from GitHub ----------------
> Url: https://github.com/KylinOLAP/Kylin/issues/101
> Created by: [Yancey1989|https://github.com/Yancey1989]
> Labels: help wanted,
> Assignee: [binmahone|https://github.com/binmahone]
> Created at: Tue Nov 25 14:22:13 CST 2014
> State: closed
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)