You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by "Junfan Zhang (Jira)" <ji...@apache.org> on 2020/11/09 14:02:00 UTC
[jira] [Comment Edited] (OOZIE-3581) Callback does not applied in
Oozie server, workflows stuk in RUNNING states.
[ https://issues.apache.org/jira/browse/OOZIE-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17228611#comment-17228611 ]
Junfan Zhang edited comment on OOZIE-3581 at 11/9/20, 2:01 PM:
---------------------------------------------------------------
Oozie uses two methods to detect the status of tasks, active and passive. Checkerservice is used to detect the status of the task. But callback will notify Oozie, and then trigger the detection. So if there is a delay, it means that the queue is backlogged.
[~gaofeng6]
was (Author: zuston):
Oozie uses two methods to detect the status of tasks, active and passive. Checkerservice is used to detect the status of the task. But callback will notify Oozie, and then trigger the detection. So if there is a delay, it means that the queue is backlogged.
> Callback does not applied in Oozie server, workflows stuk in RUNNING states.
> ----------------------------------------------------------------------------
>
> Key: OOZIE-3581
> URL: https://issues.apache.org/jira/browse/OOZIE-3581
> Project: Oozie
> Issue Type: Bug
> Components: action, workflow
> Affects Versions: 4.3.1
> Reporter: Kotsubinsky Victor
> Priority: Critical
>
> oozie version 4.3.1.3.1.0.0-78
> with HDP3.10 stack , release provides Oozie 4.3.1 and the additional Apache patches listed here: [https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.0/release-notes/content/patch_oozie.html]
> I use Hadoop kerberized cluster, run on OOZIE, YARN Mr jobs.
> 1. OOzie run mr-job via YARN
> 2. After YARN mr job completed,YYARN mr-job Successfully sent Callback request to OOzie,
> 3. in logs OOzie server, i can see this request, but OOZIE does not apply this callback request, so in WF-action-id i still see RUNNING state (until action.check process check wf-ids and swith action-id to SUCCESS state)
> LOGS in YARN:
> 2020-01-27 12:16:39,749 INFO [Thread-78] org.eclipse.jetty.util.log: Job end notification trying http://hdp3-oozie:11000/oozie/callback?id=0005607-200123121357414-oozie-oozi-W@rdb-full-table-extract-java&status=SUCCEEDED
> 2020-01-27 12:16:39,772 INFO [Thread-78] org.eclipse.jetty.util.log: Job end notification to http://hdp3-oozie:11000/oozie/callback?id=0005607-200123121357414-oozie-oozi-W@rdb-full-table-extract-java&status=SUCCEEDED succeeded
> 2020-01-27 12:16:39,772 INFO [Thread-78] org.eclipse.jetty.util.log: Job end notification succeeded for job_1579778851579_31505
>
> Oozie logs about this event:
> 2020-01-27 12:16:39,770 DEBUG CallbackServlet:526 - SERVER[hdp3-oo-2] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0005607-200123121357414-oozie-oozi-W] ACTION[0005607-200123121357414-oozie-oozi-W@rdb-full-table-extract-java] Received a CallbackServlet.doGet() with query string id=0005607-200123121357414-oozie-oozi-W@rdb-full-table-extract-java&status=SUCCEEDED
> 2020-01-27 12:16:39,776 DEBUG CompletedActionXCommand:526 - SERVER[hdp3-oo-2] USER[-] GROUP[-] TOKEN[] APP[-] JOB[0005607-200123121357414-oozie-oozi-W] ACTION[0005607-200123121357414-oozie-oozi-W@rdb-full-table-extract-java] Execute command [callback] key [null]
> 2020-01-27 12:16:39,776 DEBUG CompletedActionXCommand:526 - SERVER[hdp3-oo-2] USER[-] GROUP[-] TOKEN[] APP[-] JOB[0005607-200123121357414-oozie-oozi-W] ACTION[0005607-200123121357414-oozie-oozi-W@rdb-full-table-extract-java] Queuing [1] commands with delay [0]ms
--
This message was sent by Atlassian Jira
(v8.3.4#803005)