[jira] Commented: (PIG-411) Pig leaves HOD processes behind if Ctrl-C is used before HOD connection is fully established

["Pradeep Kamath (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12633415#action_12633415 ] 

Pradeep Kamath commented on PIG-411:
------------------------------------

Changes look good.

Couple of comments:
1)
{code}
    private synchronized void closeHod(String server){
              if (hodProcess == null){
                  // just cleanup the dir if it exists and return
                  if (hodConfDir != null)
                      deleteDir(server, hodConfDir);                                                                  
                  return;
              }                                                                                                       
              
              // hod deallocate format: hod deallocate -d <conf dir>                                                  
              String[] cmdarray = new String[4];                                                                      
              cmdarray[0] = "hod";
              cmdarray[1] = "deallocate";                                                                             
              cmdarray[2] = "-d";
             if (remoteHodConfDir != null)
                 cmdarray[3] = remoteHodConfDir;                                                                     
             else
                  cmdarray[3] = hodConfDir;                                                                           
              
              log.info("Disconnecting from HOD...");
              log.debug("Disconnect command: " + cmdToString(cmdarray));                                              
              
              try {
                  runCommand(server, cmdarray, false);                                                                
             } catch (Exception e) {
                  log.warn("Failed to disconnect from HOD; error: " + e.getMessage());                                
             } finally {
                 if (remoteHodConfDir != null)
                     deleteDir(server, remoteHodConfDir);                                                             
                 deleteDir(LOCAL, hodConfDir);                                                                        
            }                                                                                                        
            
            hodProcess = null;                                                                                       
     }

{code}

Should the code in 
{code}
if (hodProcess == null){
 ..
}
{code}

be same as the code which deletes and cleans up in 
{code}
 } finally {
                 if (remoteHodConfDir != null)
                     deleteDir(server, remoteHodConfDir);                                                             
                 deleteDir(LOCAL, hodConfDir);                                                                        
            }                    
{code}
to be consistent in the way cleanup is done?

2)
If hodProcess != null, and if we failed to deallocate hod cluster in the catch{}, should we try to destroy hodProcess -(hodProcess.destroy()) so that we don't leave the hodProcess as a zombie?

> Pig leaves HOD processes behind if Ctrl-C is used before HOD connection is fully established
> --------------------------------------------------------------------------------------------
>
>                 Key: PIG-411
>                 URL: https://issues.apache.org/jira/browse/PIG-411
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: types_branch
>            Reporter: Olga Natkovich
>            Assignee: Olga Natkovich
>            Priority: Critical
>             Fix For: types_branch
>
>         Attachments: PIG-411.patch
>
>
> The code in doHod in backend/hadoop/executionengine/HExecutionEngine.java waits for hod connection to be fully established before setting a flag that indicates need to discunnect. The flag should be set as soon as HOD process is created so that we destroy it. 
> This change will not guarantee that process is not left behind but would make it much less likely

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.