[jira] Commented: (HADOOP-3546) TaskTracker re-initialization gets stuck in cleaning up

["Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-3546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12604807#action_12604807 ] 

Amareshwari Sriramadasu commented on HADOOP-3546:
-------------------------------------------------

The problem is added by the code for Joining cleanup thread through HADOOP-2393.
TaskTracker is waiting for the task cleanup thread to come out. But taskcleanup thread comes out only when the flag _shuttingDown_ is set true. So, the tasktracker is stuck in close().

After analysing the code in TaskTracker.java, these following improvements in the code will actually fix the bug,  
1. _taskCleanupThread_ is started by static code block. That should be moved to _initialize()_. Since we want to stop the thread and start again in the reintialization. 
2. There are two flags _running_ and _shuttingDown_. _running_ is set true in _intialize()_ once all the data structures are intialized , all the threads and servers are started etc.. And _shuttingDown_ is set in the method _shutdown()_ .  _shutdown()_ is nothing but  _close()_  and stops the StatusHttpServer. So, we can move _shuttingDown_ to _close()_ and rename it to _closing_.
3.The following code in TaskTracker.run() is not necessary:
 {noformat}  if (shuttingDown) { return; } {noformat}
Since the flag is never set in the loop. 
4. _intialize()_ is called even for Denied tasktracker before shutting down (easily reproducible). So, We can call _intialize()_ only if task tracker is stale state.
5. The method _isRunning()_ (introduced in HADOOP-2393) can be removed and the flag can be directly accessed from the thread.


> TaskTracker re-initialization gets stuck in cleaning up
> -------------------------------------------------------
>
>                 Key: HADOOP-3546
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3546
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.18.0
>            Reporter: Amareshwari Sriramadasu
>            Assignee: Amareshwari Sriramadasu
>            Priority: Blocker
>             Fix For: 0.18.0
>
>
> If TaskTracker gets reinit action, it is stuck in joining task cleanup thread. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.