You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "QiangZhang (Jira)" <ji...@apache.org> on 2020/07/23 07:21:00 UTC
[jira] [Commented] (KYLIN-4657) dead-loop in
org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork
[ https://issues.apache.org/jira/browse/KYLIN-4657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17163279#comment-17163279 ]
QiangZhang commented on KYLIN-4657:
-----------------------------------
h2. some evidence：
!image-2020-07-23-15-08-45-493.png!
h2. log：
h3. First time
|java.io.IOException: Job status not available java.io.IOException: Job status not available at org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:334) at org.apache.hadoop.mapreduce.Job.getStatus(Job.java:341) at org.apache.kylin.engine.mr.common.HadoopJobStatusChecker.checkStatus(HadoopJobStatusChecker.java:38) at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:153) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)2020-07-22 04:24:08,619 ERROR [Scheduler 102132144 Job 685c0f4b-f260-8c41-2cdd-0df228e748ef-357] common.MapReduceExecutable:198 : error execute MapReduceExecutable\{id=685c0f4b-f260-8c41-2cdd-0df228e748ef-02, name=Extract Fact Table Distinct Columns, state=RUNNING}java.lang.NullPointerException at org.apache.hadoop.mapreduce.Job.getTrackingURL(Job.java:380) at org.apache.kylin.engine.mr.common.HadoopCmdOutput.getInfo(HadoopCmdOutput.java:66) at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:163) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)|
h3. second time
|2020-07-22 04:24:08,619 INFO [Scheduler 102132144 Job 685c0f4b-f260-8c41-2cdd-0df228e748ef-357] execution.AbstractExecutable:162 : Retry 1
2020-07-22 04:24:08,704 INFO [Scheduler 102132144 Job 685c0f4b-f260-8c41-2cdd-0df228e748ef-357] client.ConfiguredRMFailoverProxyProvider:100 : Failing over to rm3238
2020-07-22 04:24:08,705 INFO [Scheduler 102132144 Job 685c0f4b-f260-8c41-2cdd-0df228e748ef-357] mapred.ClientServiceDelegate:277 : Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server
2020-07-22 04:24:09,702 INFO [Scheduler 102132144 Job 685c0f4b-f260-8c41-2cdd-0df228e748ef-357] common.MapReduceExecutable:121 : mr_job_id:job_1589279266024_4691639 resumed|
h2. debug：
!image-2020-07-23-15-20-05-764.png!
> dead-loop in org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork
> -------------------------------------------------------------------------
>
> Key: KYLIN-4657
> URL: https://issues.apache.org/jira/browse/KYLIN-4657
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
> Affects Versions: all
> Reporter: QiangZhang
> Priority: Major
> Attachments: image-2020-07-23-14-54-02-079.png, image-2020-07-23-14-58-49-140.png, image-2020-07-23-15-08-45-493.png, image-2020-07-23-15-18-48-058.png, image-2020-07-23-15-20-05-764.png
>
>
> ||issue||
> |Kylin job in step3("Extract Fact Table Distinct Columns") is always running|
>
> ----
> 1.when this job is run for the first time：
> !image-2020-07-23-14-54-02-079.png!
> 2.If user has enable Job Retry (_+kylin.job.retry+_), so job will retry again
> !image-2020-07-23-14-58-49-140.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)