[jira] [Work logged] (GOBBLIN-1744) Improve logging in null cases when querying from Helix

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/GOBBLIN-1744?focusedWorklogId=838481&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-838481 ]

ASF GitHub Bot logged work on GOBBLIN-1744:
-------------------------------------------

                Author: ASF GitHub Bot
            Created on: 10/Jan/23 23:34
            Start Date: 10/Jan/23 23:34
    Worklog Time Spent: 10m 
      Work Description: homatthew commented on code in PR #3603:
URL: https://github.com/apache/gobblin/pull/3603#discussion_r1066444438


##########
gobblin-cluster/src/main/java/org/apache/gobblin/cluster/HelixUtils.java:
##########
@@ -395,11 +392,22 @@ private static void deleteStoppedHelixJob(HelixManager helixManager, String work
    * @return a map from jobNames to their Helix Workflow Ids.
    */
   public static Map<String, String> getWorkflowIdsFromJobNames(HelixManager helixManager, Collection<String> jobNames) {
-    Map<String, String> jobNameToWorkflowId = new HashMap<>();
     TaskDriver taskDriver = new TaskDriver(helixManager);
+    return getWorkflowIdsFromJobNames(taskDriver, jobNames);
+  }
+
+  public static Map<String, String> getWorkflowIdsFromJobNames(TaskDriver taskDriver, Collection<String> jobNames) {
+    Map<String, String> jobNameToWorkflowId = new HashMap<>();
     Map<String, WorkflowConfig> workflowConfigMap = taskDriver.getWorkflows();
-    for (String workflow : workflowConfigMap.keySet()) {
-      WorkflowConfig workflowConfig = taskDriver.getWorkflowConfig(workflow);
+    for (Map.Entry<String, WorkflowConfig> entry : workflowConfigMap.entrySet()) {
+      String workflow = entry.getKey();
+      WorkflowConfig workflowConfig = entry.getValue();
+      if (workflowConfig == null) {

Review Comment:
   Made change to throw exception. I made the choice to create a new custom exception because there are other places where make API calls to ZK / Helix, and I want to use this exception to explicitly say there is a Helix issue.
   
   The caller of this API (Job scheduler) is using a retryer that will automatically retry this API call again if there is an exception.





Issue Time Tracking
-------------------

    Worklog Id:     (was: 838481)
    Time Spent: 1.5h  (was: 1h 20m)

> Improve logging in null cases when querying from Helix
> ------------------------------------------------------
>
>                 Key: GOBBLIN-1744
>                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1744
>             Project: Apache Gobblin
>          Issue Type: New Feature
>            Reporter: Matthew Ho
>            Priority: Major
>          Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> There are a few edge cases where the data in Helix returns null and we either do not handle gracefully or we aren't logging in a way that makes debugging easier.
>  * HelixAssignedParticipantCheck
>  * getWorkflowIdsFromJobNames
> Both of these cases are nulls from helix



--
This message was sent by Atlassian Jira
(v8.20.10#820010)