You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Vinod K V (JIRA)" <ji...@apache.org> on 2009/03/25 11:14:50 UTC
[jira] Created: (HADOOP-5570) TaskMemoryManager should log
process-tree's status while killing tasks.
TaskMemoryManager should log process-tree's status while killing tasks.
-----------------------------------------------------------------------
Key: HADOOP-5570
URL: https://issues.apache.org/jira/browse/HADOOP-5570
Project: Hadoop Core
Issue Type: Bug
Components: mapred
Reporter: Vinod K V
This helps a lot in debugging why a particular task has gone beyond memory limits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5570) TaskMemoryManager should log
process-tree's status while killing tasks.
Posted by "Sreekanth Ramakrishnan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sreekanth Ramakrishnan updated HADOOP-5570:
-------------------------------------------
Status: Patch Available (was: Open)
> TaskMemoryManager should log process-tree's status while killing tasks.
> -----------------------------------------------------------------------
>
> Key: HADOOP-5570
> URL: https://issues.apache.org/jira/browse/HADOOP-5570
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Vinod K V
> Attachments: HADOOP-5570-1.patch, HADOOP-5570-20090614.txt, HADOOP-5570-20090615.txt, HADOOP-5570-20090619.txt, HADOOP-5570-20090622.txt
>
>
> This helps a lot in debugging why a particular task has gone beyond memory limits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-5570) TaskMemoryManager should log
process-tree's status while killing tasks.
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod K V reassigned HADOOP-5570:
---------------------------------
Assignee: Vinod K V
> TaskMemoryManager should log process-tree's status while killing tasks.
> -----------------------------------------------------------------------
>
> Key: HADOOP-5570
> URL: https://issues.apache.org/jira/browse/HADOOP-5570
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Vinod K V
>
> This helps a lot in debugging why a particular task has gone beyond memory limits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5570) TaskMemoryManager should log
process-tree's status while killing tasks.
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod K V updated HADOOP-5570:
------------------------------
Status: Open (was: Patch Available)
> TaskMemoryManager should log process-tree's status while killing tasks.
> -----------------------------------------------------------------------
>
> Key: HADOOP-5570
> URL: https://issues.apache.org/jira/browse/HADOOP-5570
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Vinod K V
> Attachments: HADOOP-5570-20090614.txt, HADOOP-5570-20090615.txt
>
>
> This helps a lot in debugging why a particular task has gone beyond memory limits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5570) TaskMemoryManager should log
process-tree's status while killing tasks.
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod K V updated HADOOP-5570:
------------------------------
Status: Patch Available (was: Open)
> TaskMemoryManager should log process-tree's status while killing tasks.
> -----------------------------------------------------------------------
>
> Key: HADOOP-5570
> URL: https://issues.apache.org/jira/browse/HADOOP-5570
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Vinod K V
> Attachments: HADOOP-5570-20090614.txt
>
>
> This helps a lot in debugging why a particular task has gone beyond memory limits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5570) TaskMemoryManager should log
process-tree's status while killing tasks.
Posted by "Sreekanth Ramakrishnan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sreekanth Ramakrishnan updated HADOOP-5570:
-------------------------------------------
Status: Open (was: Patch Available)
> TaskMemoryManager should log process-tree's status while killing tasks.
> -----------------------------------------------------------------------
>
> Key: HADOOP-5570
> URL: https://issues.apache.org/jira/browse/HADOOP-5570
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Vinod K V
> Attachments: HADOOP-5570-20090614.txt, HADOOP-5570-20090615.txt, HADOOP-5570-20090619.txt, HADOOP-5570-20090622.txt
>
>
> This helps a lot in debugging why a particular task has gone beyond memory limits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5570) TaskMemoryManager should log
process-tree's status while killing tasks.
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710299#action_12710299 ]
Hadoop QA commented on HADOOP-5570:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12408234/HADOOP-5570-20090615.txt
against trunk revision 775492.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/345/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/345/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/345/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/345/console
This message is automatically generated.
> TaskMemoryManager should log process-tree's status while killing tasks.
> -----------------------------------------------------------------------
>
> Key: HADOOP-5570
> URL: https://issues.apache.org/jira/browse/HADOOP-5570
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Vinod K V
> Attachments: HADOOP-5570-20090614.txt, HADOOP-5570-20090615.txt
>
>
> This helps a lot in debugging why a particular task has gone beyond memory limits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5570) TaskMemoryManager should log
process-tree's status while killing tasks.
Posted by "Sreekanth Ramakrishnan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710310#action_12710310 ]
Sreekanth Ramakrishnan commented on HADOOP-5570:
------------------------------------------------
+1 to the latest patch.
> TaskMemoryManager should log process-tree's status while killing tasks.
> -----------------------------------------------------------------------
>
> Key: HADOOP-5570
> URL: https://issues.apache.org/jira/browse/HADOOP-5570
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Vinod K V
> Attachments: HADOOP-5570-20090614.txt, HADOOP-5570-20090615.txt
>
>
> This helps a lot in debugging why a particular task has gone beyond memory limits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5570) TaskMemoryManager should log
process-tree's status while killing tasks.
Posted by "Sreekanth Ramakrishnan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sreekanth Ramakrishnan updated HADOOP-5570:
-------------------------------------------
Attachment: HADOOP-5570-1.patch
Attaching patch merging with lastest trunk.
> TaskMemoryManager should log process-tree's status while killing tasks.
> -----------------------------------------------------------------------
>
> Key: HADOOP-5570
> URL: https://issues.apache.org/jira/browse/HADOOP-5570
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Vinod K V
> Attachments: HADOOP-5570-1.patch, HADOOP-5570-20090614.txt, HADOOP-5570-20090615.txt, HADOOP-5570-20090619.txt, HADOOP-5570-20090622.txt
>
>
> This helps a lot in debugging why a particular task has gone beyond memory limits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5570) TaskMemoryManager should log
process-tree's status while killing tasks.
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod K V updated HADOOP-5570:
------------------------------
Status: Open (was: Patch Available)
> TaskMemoryManager should log process-tree's status while killing tasks.
> -----------------------------------------------------------------------
>
> Key: HADOOP-5570
> URL: https://issues.apache.org/jira/browse/HADOOP-5570
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Vinod K V
> Attachments: HADOOP-5570-20090614.txt
>
>
> This helps a lot in debugging why a particular task has gone beyond memory limits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5570) TaskMemoryManager should log
process-tree's status while killing tasks.
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod K V updated HADOOP-5570:
------------------------------
Attachment: HADOOP-5570-20090614.txt
Attaching a patch to fix the issue. This patch reads /proc/pid/cmdline to get the command line of each process and uses this to get the process-tree's dump. The dump is logged both on the TaskTracker and is displayed on the web-ui in the Errors tab. Added a simple test-case and also tested this on a local cluster.
> TaskMemoryManager should log process-tree's status while killing tasks.
> -----------------------------------------------------------------------
>
> Key: HADOOP-5570
> URL: https://issues.apache.org/jira/browse/HADOOP-5570
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Vinod K V
> Attachments: HADOOP-5570-20090614.txt
>
>
> This helps a lot in debugging why a particular task has gone beyond memory limits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5570) TaskMemoryManager should log
process-tree's status while killing tasks.
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712866#action_12712866 ]
Hadoop QA commented on HADOOP-5570:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12408803/HADOOP-5570-20090622.txt
against trunk revision 778388.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/399/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/399/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/399/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/399/console
This message is automatically generated.
> TaskMemoryManager should log process-tree's status while killing tasks.
> -----------------------------------------------------------------------
>
> Key: HADOOP-5570
> URL: https://issues.apache.org/jira/browse/HADOOP-5570
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Vinod K V
> Attachments: HADOOP-5570-20090614.txt, HADOOP-5570-20090615.txt, HADOOP-5570-20090619.txt, HADOOP-5570-20090622.txt
>
>
> This helps a lot in debugging why a particular task has gone beyond memory limits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5570) TaskMemoryManager should log
process-tree's status while killing tasks.
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod K V updated HADOOP-5570:
------------------------------
Status: Patch Available (was: Open)
> TaskMemoryManager should log process-tree's status while killing tasks.
> -----------------------------------------------------------------------
>
> Key: HADOOP-5570
> URL: https://issues.apache.org/jira/browse/HADOOP-5570
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Vinod K V
> Attachments: HADOOP-5570-20090614.txt, HADOOP-5570-20090615.txt, HADOOP-5570-20090619.txt, HADOOP-5570-20090622.txt
>
>
> This helps a lot in debugging why a particular task has gone beyond memory limits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5570) TaskMemoryManager should log
process-tree's status while killing tasks.
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod K V updated HADOOP-5570:
------------------------------
Attachment: HADOOP-5570-20090615.txt
Small changes to the patch. The process-tree dump propagates back to the JobClient. So, pretty-printing the process-tree dump now to reduce clutter on the client. Also adding tip-id in the process-tree dump for quick reference.
> TaskMemoryManager should log process-tree's status while killing tasks.
> -----------------------------------------------------------------------
>
> Key: HADOOP-5570
> URL: https://issues.apache.org/jira/browse/HADOOP-5570
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Vinod K V
> Attachments: HADOOP-5570-20090614.txt, HADOOP-5570-20090615.txt
>
>
> This helps a lot in debugging why a particular task has gone beyond memory limits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5570) TaskMemoryManager should log
process-tree's status while killing tasks.
Posted by "Sreekanth Ramakrishnan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12709341#action_12709341 ]
Sreekanth Ramakrishnan commented on HADOOP-5570:
------------------------------------------------
Changes look fine to me. +1 to patch
> TaskMemoryManager should log process-tree's status while killing tasks.
> -----------------------------------------------------------------------
>
> Key: HADOOP-5570
> URL: https://issues.apache.org/jira/browse/HADOOP-5570
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Vinod K V
> Attachments: HADOOP-5570-20090614.txt
>
>
> This helps a lot in debugging why a particular task has gone beyond memory limits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5570) TaskMemoryManager should log
process-tree's status while killing tasks.
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod K V updated HADOOP-5570:
------------------------------
Attachment: HADOOP-5570-20090619.txt
Another small change to the patch. This adds to the dump the memory usage of each process in the process-tree at the time of killing.
> TaskMemoryManager should log process-tree's status while killing tasks.
> -----------------------------------------------------------------------
>
> Key: HADOOP-5570
> URL: https://issues.apache.org/jira/browse/HADOOP-5570
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Vinod K V
> Attachments: HADOOP-5570-20090614.txt, HADOOP-5570-20090615.txt, HADOOP-5570-20090619.txt
>
>
> This helps a lot in debugging why a particular task has gone beyond memory limits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5570) TaskMemoryManager should log
process-tree's status while killing tasks.
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod K V updated HADOOP-5570:
------------------------------
Attachment: HADOOP-5570-20090622.txt
Patch fixing the test case.
> TaskMemoryManager should log process-tree's status while killing tasks.
> -----------------------------------------------------------------------
>
> Key: HADOOP-5570
> URL: https://issues.apache.org/jira/browse/HADOOP-5570
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Vinod K V
> Attachments: HADOOP-5570-20090614.txt, HADOOP-5570-20090615.txt, HADOOP-5570-20090619.txt, HADOOP-5570-20090622.txt
>
>
> This helps a lot in debugging why a particular task has gone beyond memory limits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5570) TaskMemoryManager should log
process-tree's status while killing tasks.
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod K V updated HADOOP-5570:
------------------------------
Status: Patch Available (was: Open)
> TaskMemoryManager should log process-tree's status while killing tasks.
> -----------------------------------------------------------------------
>
> Key: HADOOP-5570
> URL: https://issues.apache.org/jira/browse/HADOOP-5570
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Vinod K V
> Assignee: Vinod K V
> Attachments: HADOOP-5570-20090614.txt, HADOOP-5570-20090615.txt
>
>
> This helps a lot in debugging why a particular task has gone beyond memory limits.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.