You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Vinod K V (JIRA)" <ji...@apache.org> on 2009/03/13 14:39:50 UTC
[jira] Created: (HADOOP-5488) HADOOP-2721 doesn't clean up
descendant processes of a jvm that exits cleanly after running a task
successfully
HADOOP-2721 doesn't clean up descendant processes of a jvm that exits cleanly after running a task successfully
---------------------------------------------------------------------------------------------------------------
Key: HADOOP-5488
URL: https://issues.apache.org/jira/browse/HADOOP-5488
Project: Hadoop Core
Issue Type: Bug
Components: mapred
Reporter: Vinod K V
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5488) HADOOP-2721 doesn't clean up
descendant processes of a jvm that exits cleanly after running a task
successfully
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12696001#action_12696001 ]
Hemanth Yamijala commented on HADOOP-5488:
------------------------------------------
bq. changing the methodCall umbilical.getTask(JVMId) to umbilical.getTask(JVMId, pid) in Child.java
It seems cleaner to define a new method: umbilical.setProcessHandle(JVMId, handle). This way we send this information only once per JVM. We can also handle cases like what happens if this call fails. For e.g., I suppose this call will be made before the child runs the task, and if this call fails, it should ideally abort.
> HADOOP-2721 doesn't clean up descendant processes of a jvm that exits cleanly after running a task successfully
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5488
> URL: https://issues.apache.org/jira/browse/HADOOP-5488
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Ravi Gummadi
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5488) HADOOP-2721 doesn't clean up
descendant processes of a jvm that exits cleanly after running a task
successfully
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12702079#action_12702079 ]
Hudson commented on HADOOP-5488:
--------------------------------
Integrated in Hadoop-trunk #815 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/815/])
> HADOOP-2721 doesn't clean up descendant processes of a jvm that exits cleanly after running a task successfully
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5488
> URL: https://issues.apache.org/jira/browse/HADOOP-5488
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Ravi Gummadi
> Fix For: 0.21.0
>
> Attachments: HADOOP-5488.patch, HADOOP_5488_5614.patch, HADOOP_5488_5614.v1.2.patch, HADOOP_5488_5614.v1.3.patch, HADOOP_5488_5614.v1.patch, testcase5488.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5488) HADOOP-2721 doesn't clean up
descendant processes of a jvm that exits cleanly after running a task
successfully
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699619#action_12699619 ]
Vinod K V commented on HADOOP-5488:
-----------------------------------
This patch clashes with HADOOP-4490 which is almost close to commit. This patch should wait for HADOOP-4490 and the changes should be merged.
> HADOOP-2721 doesn't clean up descendant processes of a jvm that exits cleanly after running a task successfully
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5488
> URL: https://issues.apache.org/jira/browse/HADOOP-5488
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Ravi Gummadi
> Attachments: HADOOP-5488.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5488) HADOOP-2721 doesn't clean up
descendant processes of a jvm that exits cleanly after running a task
successfully
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Gummadi updated HADOOP-5488:
---------------------------------
Attachment: HADOOP_5488_5614.v1.3.patch
Attaching new patch with some javadoc change.
Unit tests passed on my linux machine. No new failures in unit tests on Windows machine.
Reliability test passed.
ant test-patch gave:
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 12 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
> HADOOP-2721 doesn't clean up descendant processes of a jvm that exits cleanly after running a task successfully
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5488
> URL: https://issues.apache.org/jira/browse/HADOOP-5488
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Ravi Gummadi
> Attachments: HADOOP-5488.patch, HADOOP_5488_5614.patch, HADOOP_5488_5614.v1.2.patch, HADOOP_5488_5614.v1.3.patch, HADOOP_5488_5614.v1.patch, testcase5488.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5488) HADOOP-2721 doesn't clean up
descendant processes of a jvm that exits cleanly after running a task
successfully
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Gummadi updated HADOOP-5488:
---------------------------------
Attachment: HADOOP_5488_5614.v1.2.patch
Incorpoated Vinod's offline comments and syncing with the trunk, attaching new patch.
Please review and provide your comments.
> HADOOP-2721 doesn't clean up descendant processes of a jvm that exits cleanly after running a task successfully
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5488
> URL: https://issues.apache.org/jira/browse/HADOOP-5488
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Ravi Gummadi
> Attachments: HADOOP-5488.patch, HADOOP_5488_5614.patch, HADOOP_5488_5614.v1.2.patch, HADOOP_5488_5614.v1.patch, testcase5488.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5488) HADOOP-2721 doesn't clean up
descendant processes of a jvm that exits cleanly after running a task
successfully
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12681711#action_12681711 ]
Vinod K V commented on HADOOP-5488:
-----------------------------------
HADOOP-2721 incorporated changes to clean up descendant processes only in the code path of killing of a task. And when a jvm that successfully ran a task exits, the pidfile gets removed and so even if a KillTaskAction is subsequently received, descendants will not be cleaned up for the lack of the root pid.
> HADOOP-2721 doesn't clean up descendant processes of a jvm that exits cleanly after running a task successfully
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5488
> URL: https://issues.apache.org/jira/browse/HADOOP-5488
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5488) HADOOP-2721 doesn't clean up
descendant processes of a jvm that exits cleanly after running a task
successfully
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12701459#action_12701459 ]
Vinod K V commented on HADOOP-5488:
-----------------------------------
+1 for the patch.
> HADOOP-2721 doesn't clean up descendant processes of a jvm that exits cleanly after running a task successfully
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5488
> URL: https://issues.apache.org/jira/browse/HADOOP-5488
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Ravi Gummadi
> Attachments: HADOOP-5488.patch, HADOOP_5488_5614.patch, HADOOP_5488_5614.v1.2.patch, HADOOP_5488_5614.v1.3.patch, HADOOP_5488_5614.v1.patch, testcase5488.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5488) HADOOP-2721 doesn't clean up
descendant processes of a jvm that exits cleanly after running a task
successfully
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700127#action_12700127 ]
Vinod K V commented on HADOOP-5488:
-----------------------------------
Gone through the patch. Looks good overall. Some code comments:
- JvmContext : Constructor should just take a pid and child jvm should pass it after reading it from the environment.
- The 'isProcessGroup' check is not needed in ProcessTree.sigKillInCurrentThread() as we may want to send SIGKILL to processes not started as sessions also.
- Can you please remove the unused import in DefaultTaskController that was perhaps mistakenly left in HADOOP-4490:
import org.apache.hadoop.fs.Path;
- No need for changing the controller.killTaskJVM() to return a boolean. If jvmmanager.kill() has to be reentrant it can just maintain the variables.
-- Is it needed to make kill to be reentrant now?
- I think DefaultTaskController.killTaskJvm() method should be better commented, especially the various conditionals.
- Fix the LOG var of the abstract class Task (HADOOP-5351)
- TaskLog: The place where the comment says pid files are not used anymore, we shoujld also suggest the method that should instead be used directly.
- There should be a comment about the bump up of the version number of TaskUmbilicalProtocol saying there is a change in signature of getTask()
- TestKillSubProcesses:
-- Most of the methods in this class can be private instead of package-private unless explicitly needed.
-- I see some hardcoded values 5 in the code like in "scriptDir.toString().substring(5)". This is perhaps related to parsing out the filesystem scheme info. We should instead use URI methods to get the actual path name.
-- KillingMapperWithChildren, FailingMapperWithChildren and SucceedMapperWithChildren can be futher refactored.
-- Can we use some job in UtilsForTest instead of our own runJob() ?
- Is this possible to test if this test-case fails without the changes in the patch and succeeds with the changes?
> HADOOP-2721 doesn't clean up descendant processes of a jvm that exits cleanly after running a task successfully
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5488
> URL: https://issues.apache.org/jira/browse/HADOOP-5488
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Ravi Gummadi
> Attachments: HADOOP-5488.patch, HADOOP_5488_5614.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-5488) HADOOP-2721 doesn't clean up
descendant processes of a jvm that exits cleanly after running a task
successfully
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Gummadi reassigned HADOOP-5488:
------------------------------------
Assignee: Ravi Gummadi
> HADOOP-2721 doesn't clean up descendant processes of a jvm that exits cleanly after running a task successfully
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5488
> URL: https://issues.apache.org/jira/browse/HADOOP-5488
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Ravi Gummadi
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5488) HADOOP-2721 doesn't clean up
descendant processes of a jvm that exits cleanly after running a task
successfully
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Gummadi updated HADOOP-5488:
---------------------------------
Attachment: HADOOP-5488.patch
Attaching patch with code changes for killing subprocesses of successful tasks and failed tasks. Usage of pid files is removed and pid is exported to an env variable and then sent from taskJvm to TT using umbilical protocol, which will be used for killing subprocesses of tasks.
Please review and provide your comments.
> HADOOP-2721 doesn't clean up descendant processes of a jvm that exits cleanly after running a task successfully
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5488
> URL: https://issues.apache.org/jira/browse/HADOOP-5488
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Ravi Gummadi
> Attachments: HADOOP-5488.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5488) HADOOP-2721 doesn't clean up
descendant processes of a jvm that exits cleanly after running a task
successfully
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12681827#action_12681827 ]
Devaraj Das commented on HADOOP-5488:
-------------------------------------
In addition to successful tasks, tasks that fail should also be taken care of.
> HADOOP-2721 doesn't clean up descendant processes of a jvm that exits cleanly after running a task successfully
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5488
> URL: https://issues.apache.org/jira/browse/HADOOP-5488
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5488) HADOOP-2721 doesn't clean up
descendant processes of a jvm that exits cleanly after running a task
successfully
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Gummadi updated HADOOP-5488:
---------------------------------
Attachment: testcase5488.patch
Attaching testcase that fails with trunk because of subprocesses not getting killed for successful and failing tasks.
> HADOOP-2721 doesn't clean up descendant processes of a jvm that exits cleanly after running a task successfully
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5488
> URL: https://issues.apache.org/jira/browse/HADOOP-5488
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Ravi Gummadi
> Attachments: HADOOP-5488.patch, HADOOP_5488_5614.patch, HADOOP_5488_5614.v1.patch, testcase5488.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5488) HADOOP-2721 doesn't clean up
descendant processes of a jvm that exits cleanly after running a task
successfully
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Gummadi updated HADOOP-5488:
---------------------------------
Attachment: HADOOP_5488_5614.v1.patch
Attaching patch with the code changes suggested by Vinod.
Please review and provide your comments.
> HADOOP-2721 doesn't clean up descendant processes of a jvm that exits cleanly after running a task successfully
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5488
> URL: https://issues.apache.org/jira/browse/HADOOP-5488
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Ravi Gummadi
> Attachments: HADOOP-5488.patch, HADOOP_5488_5614.patch, HADOOP_5488_5614.v1.patch, testcase5488.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-5488) HADOOP-2721 doesn't clean up
descendant processes of a jvm that exits cleanly after running a task
successfully
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das resolved HADOOP-5488.
---------------------------------
Resolution: Fixed
Fix Version/s: 0.21.0
Hadoop Flags: [Reviewed]
I just committed this. Thanks, Ravi!
> HADOOP-2721 doesn't clean up descendant processes of a jvm that exits cleanly after running a task successfully
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5488
> URL: https://issues.apache.org/jira/browse/HADOOP-5488
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Ravi Gummadi
> Fix For: 0.21.0
>
> Attachments: HADOOP-5488.patch, HADOOP_5488_5614.patch, HADOOP_5488_5614.v1.2.patch, HADOOP_5488_5614.v1.3.patch, HADOOP_5488_5614.v1.patch, testcase5488.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5488) HADOOP-2721 doesn't clean up
descendant processes of a jvm that exits cleanly after running a task
successfully
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12695999#action_12695999 ]
Ravi Gummadi commented on HADOOP-5488:
--------------------------------------
For successful tasks and failed tasks, as JvmRunner.kill() is not getting called, their subprocesses are not killed. Even if we call kill for these tasks, the cleanup of pidFile might be done already and we can't get pid of taskJvm.
So I am planning to export pid of task in some env variable and then sending pid of taskJvm from Child(task) to parent(TT) using umbilical protocol ---- changing the methodCall umbilical.getTask(JVMId) to umbilical.getTask(JVMId, pid) in Child.java and getTask() in TT would set pid in jvmIdToPidMap, which will be used by JvmRunner.kill(). And JvmRunner.runChild() will call kill() in the finally block, if it was not called earlier for this jvm.
This approach would avoid the pidFiles altogether.
Thoughts ?
> HADOOP-2721 doesn't clean up descendant processes of a jvm that exits cleanly after running a task successfully
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5488
> URL: https://issues.apache.org/jira/browse/HADOOP-5488
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Ravi Gummadi
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5488) HADOOP-2721 doesn't clean up
descendant processes of a jvm that exits cleanly after running a task
successfully
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699999#action_12699999 ]
Devaraj Das commented on HADOOP-5488:
-------------------------------------
Sorry for not looking at the patch earlier. But I think the RPC getTask(JVMId) could simply be enhanced to take a (new class object) JVMContext, i.e., getTask(JVMContext). The class could have the fields as PID and the JVMId, and in the case of windows, the PID string is empty.
> HADOOP-2721 doesn't clean up descendant processes of a jvm that exits cleanly after running a task successfully
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5488
> URL: https://issues.apache.org/jira/browse/HADOOP-5488
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Ravi Gummadi
> Attachments: HADOOP-5488.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5488) HADOOP-2721 doesn't clean up
descendant processes of a jvm that exits cleanly after running a task
successfully
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Gummadi updated HADOOP-5488:
---------------------------------
Attachment: HADOOP_5488_5614.patch
Incorporated Devaraj's comment.
Attaching combined patch of 5488 and 5614 as there are some clashing code changes.
Please review and provide your comments.
> HADOOP-2721 doesn't clean up descendant processes of a jvm that exits cleanly after running a task successfully
> ---------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5488
> URL: https://issues.apache.org/jira/browse/HADOOP-5488
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Ravi Gummadi
> Attachments: HADOOP-5488.patch, HADOOP_5488_5614.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.