[GitHub] [incubator-gobblin] sv2000 commented on a change in pull request #2892: GOBBLIN-1052: Create a spec consumer path if it does not exist in FS …

[GitBox <gi...@apache.org>]

sv2000 commented on a change in pull request #2892: GOBBLIN-1052: Create a spec consumer path if it does not exist in FS …
URL: https://github.com/apache/incubator-gobblin/pull/2892#discussion_r383041046
 
 

 ##########
 File path: gobblin-cluster/src/main/java/org/apache/gobblin/cluster/HelixUtils.java
 ##########
 @@ -293,6 +294,10 @@ private static void deleteStoppedHelixJob(HelixManager helixManager, String work
     Map<String, WorkflowConfig> workflowConfigMap = taskDriver.getWorkflows();
     for (String workflow : workflowConfigMap.keySet()) {
       WorkflowConfig workflowConfig = taskDriver.getWorkflowConfig(workflow);
+      //Filter out any stale Helix workflows which are not running.
+      if (workflowConfig.getTargetState() != TargetState.START) {
 
 Review comment:
   Thanks for this comment. The purpose of this method is two fold: 1. The caller may want to cancel a Helix workflow given a job name and 2. The caller may want to get the workflow id and use that to query other information such as the workunits processed by the job. For both 1 and 2, the caller is only interested in the active workflows. In fact, the stale workflows are auto-pruged by Helix based on an expiry interval.
   
   I have modified the javadoc for this method to be more explicit about which workflows are returned.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services