You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gobblin.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2021/11/04 19:38:00 UTC
[jira] [Work logged] (GOBBLIN-1552) fix flow status reporting when
dag manager is not enabled
[ https://issues.apache.org/jira/browse/GOBBLIN-1552?focusedWorklogId=676672&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-676672 ]
ASF GitHub Bot logged work on GOBBLIN-1552:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 04/Nov/21 19:37
Start Date: 04/Nov/21 19:37
Worklog Time Spent: 10m
Work Description: arjun4084346 commented on a change in pull request #3403:
URL: https://github.com/apache/gobblin/pull/3403#discussion_r743146832
##########
File path: gobblin-runtime/src/main/java/org/apache/gobblin/service/monitoring/JobStatusRetriever.java
##########
@@ -218,4 +228,42 @@ public static boolean isFlowStatus(org.apache.gobblin.service.monitoring.JobStat
return jobStatus.getJobName() != null && jobStatus.getJobGroup() != null
&& jobStatus.getJobName().equals(JobStatusRetriever.NA_KEY) && jobStatus.getJobGroup().equals(JobStatusRetriever.NA_KEY);
}
+
+ public static ExecutionStatus getFlowStatusFromJobStatuses(boolean dagManagerEnabled, Iterator<JobStatus> jobStatusIterator) {
+ ExecutionStatus flowExecutionStatus = ExecutionStatus.$UNKNOWN;
+
+ if (dagManagerEnabled) {
+ while (jobStatusIterator.hasNext()) {
+ JobStatus jobStatus = jobStatusIterator.next();
+ // Check if this is the flow status instead of a single job status
+ if (JobStatusRetriever.isFlowStatus(jobStatus)) {
+ flowExecutionStatus = ExecutionStatus.valueOf(jobStatus.getEventName());
+ }
+ }
+ } else {
+ Set<ExecutionStatus> jobStatuses = new HashSet<>();
+ while (jobStatusIterator.hasNext()) {
+ JobStatus jobStatus = jobStatusIterator.next();
+ // because in absence of DagManager we do not get all flow level events, we will ignore the flow level events
+ // we actually get and purely calculate flow status based on flow statuses.
Review comment:
in future, i actually plan to remove the 'dagManagerDisabled' blocks :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 676672)
Time Spent: 1.5h (was: 1h 20m)
> fix flow status reporting when dag manager is not enabled
> ---------------------------------------------------------
>
> Key: GOBBLIN-1552
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1552
> Project: Apache Gobblin
> Issue Type: Bug
> Reporter: Arjun Singh Bora
> Priority: Major
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> flow status is determined by looking at the flow level events. but flow level events are not emitted outside of dag manager. so currently flow status is not being determined correctly when dag manager is disabled
--
This message was sent by Atlassian Jira
(v8.3.4#803005)