[GitHub] [helix] alirezazamani commented on a change in pull request #1508: Fix ondemand rebalance flooding and log flooding caused by dangling jobs

[GitBox <gi...@apache.org>]

alirezazamani commented on a change in pull request #1508:
URL: https://github.com/apache/helix/pull/1508#discussion_r516829905



##########
File path: helix-core/src/main/java/org/apache/helix/common/caches/TaskDataCache.java
##########
@@ -116,33 +116,6 @@ public synchronized boolean refresh(HelixDataAccessor accessor,
       }
     }
 
-    // The following 3 blocks is for finding a list of workflows whose JobDAGs have been changed
-    // because their RuntimeJobDags would need to be re-built
-    // newly added jobs
-    for (String jobName : newJobConfigs.keySet()) {
-      if (!_jobConfigMap.containsKey(jobName) && newJobConfigs.get(jobName).getWorkflow() != null) {
-        workflowsUpdated.add(newJobConfigs.get(jobName).getWorkflow());
-      }
-
-      // Only for JobQueues when a new job is enqueued, there exists a race condition where only
-      // JobConfig is updated and the RuntimeJobDag does not get updated because when the client
-      // (TaskDriver) submits, it creates JobConfig ZNode first and modifies its parent JobDag next.
-      // To ensure that they are both properly updated, check that workflow's DAG and existing
-      // JobConfigs are consistent for JobQueues
-      JobConfig jobConfig = newJobConfigs.get(jobName);
-      if (_workflowConfigMap.containsKey(jobConfig.getWorkflow())) {
-        WorkflowConfig workflowConfig = _workflowConfigMap.get(jobConfig.getWorkflow());
-        // Check that the job's parent workflow's DAG contains this job
-        if ((workflowConfig.isJobQueue() || !workflowConfig.isTerminable()) && !_runtimeJobDagMap
-            .get(workflowConfig.getWorkflowId()).getAllNodes().contains(jobName)) {
-          // Inconsistency between JobConfigs and DAGs found. Add the workflow to workflowsUpdated
-          // to rebuild the RuntimeJobDag
-          workflowsUpdated.add(jobConfig.getWorkflow());
-        }
-      }
-    }
-
-    // Removed jobs
     // This block makes sure that the workflow config has been changed.
     // This avoid the race condition where job config has been purged but job has not been deleted

Review comment:
       Maybe this comment needs to be changed. It was added for the purge race condition. Now it is serving as general-purpose workflow config modification.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org